Sysadmin Day Q&A with Kirk Schnable

Being a sysadmin is no easy feat, especially for sysadmins working for a hosting provider like GigeNET. They’re tasked with overseeing the software and network of hundreds of our clients all over the world, working day and night to ensure our customer’s systems are up and running. Unlike many other professions, there’s not one clear path to becoming a sysadmin.
I sat down with one of our sysadmin experts, Kirk, to learn all about how to become a sysadmin, why he’s so passionate about system administration, and his advice to future sysadmin.

Q: How did you discover your love for technology? When did you know you wanted to become a sysadmin?

A: I honestly don’t remember a time when I was not interested in technology.
I started messing around with old MS-DOS computers when I was in about second grade and was playing games on my mom’s old Apple IIe long before that. It was in grade school that I started to experiment with programming in BASIC and taking an interest in learning how things worked and how to make computers work for me. I was learning on second-hand hardware and software which was old at the time, but I think that learning on older systems taught me more low-level skills and understanding.
I feel like if I had started later, with newer and more “graphical” operating systems, that I wouldn’t have had the opportunity to learn as much. I think that learning so early in life may have played a great role in making technology so second nature to me as well.
My mom has an anecdote about taking me to the library as a kid to go on the Internet and explore websites, before the Internet was very mainstream in our area in homes, but around the time that kids shows were advertising things like their “AOL keywords.” This was before I could even read. She told me that she thinks a huge motivating factor for me learning how to read early on was wanting to read things on the computer without her help.
I think this has been true for a lot of things, that my passion for technology has motivated me in other ways too.

Q: How did you become a sysadmin?

A: In my junior year of high school, I got in touch with the tech department and started to volunteer. At the time, their tech department was a one-man show. This was my first real professional experience with working in a tech department.
When I graduated and set up my plans for college, the high school hired me on part time and I continued to work for their tech department, which became my first tech job. I guess you could technically have called me a sysadmin at that time.
In one of my computer science classes in college, I met Sara, who at the time was working in a development role at GigeNET, and she brought this company to my attention and put me in touch with management here. The rest as they say, is history!

Q: You’ve been with GigeNET for a few years now. What does a typical day as a sysadmin look like?

A: It is difficult to describe a typical day, I often do not know what I’ll be walking in to each day.
A large part of the responsibility of GigeNET’s sysadmins is assisting with customer requests, and that is inherently very unpredictable. Some days may be relatively quiet, and others you may unexpectedly need to migrate several servers or take on some other large project for a client.
We also take on internal projects, everything from deploying new servers and infrastructure in our datacenters to managing our inventory and testing and recycling used hardware.
I personally also handle abuse complaints, so I work with our customers regularly to help them address issues with abuse matters like phishing, compromised servers, and copyright infringement complaints.
I also handle any legal compliance documentation that we receive so I will occasionally interact with the police, FBI, and other agencies to comply with law enforcement requests.
We do a lot of varied things here! The variation is partially why I enjoy my role here, it definitely doesn’t get boring.

Q. I’ve heard that our customers love working with you. If they call in or submit a support ticket, you’re always insightful and provide practical ways to make their systems better. Do you like this part of the job or would you rather stay behind the scenes?

A: I enjoy being on the front lines and interacting with customers. Of course there are occasionally some customers who are difficult to work with, but overall the customer service aspect is one of the most fulfilling parts of this role for me.
As a technical person myself, I’ve had no shortage of bad customer service experiences from tech companies. For example I’ve been in situations at home in the past where there have been issues with my Internet service, and convincing the ISP of what I believe the problem is (and that it’s not on my end) is an uphill battle.
At those companies, I’ve usually found one or two staff members who I can eventually get through to and then who act as a huge asset to me moving forward. This is the honest truth – I once left an ISP because my contact there left his job, the service deteriorated, and I couldn’t get anyone to listen to me anymore and could never get the issues resolved.
[quote_colored name=”” icon_quote=”no”]So I strive to be that person who someone else is glad to have pick up their call/ticket, I like knowing that a customer who knows me may feel a trust that their issue will be resolved completely because they see me working on it. [/quote_colored]I know how it feels to be on the other end of that relationship and that it can really make a difference in someone’s life. That’s why I love the customer service part of the role and find it so fulfilling.

Q: What does your workspace set up look like?

A: I keep my desk very clean and spartan, I do the same at home actually. There’s not much on my desk at any given time besides my equipment. I keep a laptop and tablet charger here always so that I can quickly dock my mobile stuff when I get in the office.
My work computer is running Linux Mint with the MATE desktop interface, and I have 3 monitors attached on a monitor tree. My monitor tree is relatively high, allowing me to open my laptop on my desk without obstructing any of my workstation screens.
Usually my workflow is along the lines of: Skype, music, and a notepad on the left screen, research, any browser tabs, connections to any servers I’m working on, and any currently open documents on the middle screen, and always watching the ticket queue on the right screen.
Having a lot of screen real-estate really makes all the difference, especially when things get busy and I have multiple tickets and multiple customer servers open at the same time.
Since I work second shift and am often by myself, I brought in some nice speakers with a subwoofer so I can really turn it up after everyone heads home for the night. Music is very important and motivating to me. The right tunes can really put me in the zone sometimes.

Q: Now, I heard you have a pretty cool computer setup at home. What’s that like?

A: My setup at home is actually very similar to my setup at work, but with a few additions.
At my desk at home, I often use two computers. I use a program called Synergy to integrate their keyboard and mouse together, so that I can control both computers as if they are a single computer.
My main desktop at home is also running Linux Mint with the MATE desktop UI. I use a monitor tree at home similar to the one I have at work, also with 3 monitors, but my monitors at home are a bit larger. I use this computer most of the time for anything that I do online while I’m at home, from research to shopping to YouTube.
Above my Linux monitor tree, I mounted a 43” 4K TV which is attached to my Windows 10 gaming PC. I don’t keep this PC on all the time as it uses a fairly large amount of power and can really heat up the room, but I turn this on when I want to play some games. I have a rather large Steam library and recently upgraded this computer’s graphics card to a 1080TI, so it’s quite capable.
A picture is worth 1000 words:

I have put a lot of time and effort into my network at home. I have installed multiple ethernet runs to most rooms in my house which go back to centralized gigabit switches. I recently installed 7 IP cameras for video surveillance around my house as well. I have 3 WiFi access points, 2 of which are mounted in the attic and 1 of which is mounted in the garage rafters. These provide pretty good wireless coverage throughout my house as well as my front and back yard.
I also have my own little mini-datacenter, a 24U APC Netshelter server rack currently containing my custom built file server and 3 HP servers I picked up on eBay. I use the file server to store all of my data, backups, media, etc. The HP servers are more for “compute” than storage, and currently all 3 of them are VMware servers running virtual machines for my various internal services and projects.
In the future, I would like to look at some fiber projects, especially for the large things like my server rack, but currently everything in my house is gigabit ethernet. Since my uplink to the Internet is 500Mbps, this is fine (for now). 😉

Q: So I take it being a sysadmin is a hobby as well as a job for you? What are your other hobbies?

A: I’m sure you got that sense from my answer to the last question as well. 😛
A lot of my hobbies do revolve around technology, but in other ways.
I’ve spent a lot of time in the past experimenting with wireless. I wouldn’t call this “sysadmin” and it’s definitely not something I do at work.
I have a lot of high powered wireless gear and antennas and even a 2.4GHz spectrum analyzer around from past projects. I always found long-range wireless to be fascinating, and back when I lived with my parents in the more rural countryside where there were a lot of open fields, playing around with long range wireless was fairly practical. Not so much now that I live in the suburbs.
This became an interest of mine because this was how we got broadband at my parents’ house for years (in fact their broadband connection is still a long range wireless connection to a tower several miles away) so I wanted to know all the ins and outs so that I could troubleshoot and understand my connection and how to care for it when it was having issues.
All of this said – I do not like wireless. It is a necessary evil sometimes, but wired will always be faster and more reliable, hence why I put so much effort into wiring my house.
Of course I am also a gamer, although often this takes a back-seat as it feels “not productive” to me so I don’t spend a lot of time gaming usually. I only game on PC and have never had much of an interest in consoles, other than the original Nintendo Entertainment System, which was the first gaming system I ever owned. I still love Super Mario Bros. 3, but I usually play it on an emulator on my PC these days.
Along with the gaming hobby, I have been streaming on Twitch for the past several years. I started out by just streaming games I was playing in order to have a social experience of sharing them with friends, but it started to tie into my next thing:
About a year ago, I picked up my first professional DJ equipment and started to learn about mixing. I streamed most of my learning experience on Twitch, and eventually started to get a following from that. I still have a lot to learn, but I really enjoy mixing music and have been doing DJ sets on Twitch every weekend as much as possible. My favorite genres to mix are the type of stuff you would hear at an EDM festival, like dubstep and trap.

Q: I love reading your blogs because you’re always discovering new technologies and writing on how to use them. How do you find these and figure out what works best?

A: A lot of this just comes down to Google and involvement in the open source community. I’ve found that for every problem there are usually a handful of projects out there trying to solve it, and then it’s just about finding the best one for my use case. That can be the tricky part as usually none of the projects are perfect, and each one has a community surrounding it who will often blindly swear that their project is the best.
Usually I’ll look for some projects that do what I’m looking for, check out some documentation and see how good that is, and then maybe install 2 or 3 different ones and try them out. This is where my VM server at home comes in handy, because I’ll usually spin up separate test environments for each thing and then trash the ones I didn’t use.
For example when I found Syncthing, I also tried out Seafile and BitTorrent Sync (which no longer exists as an open source product now). I liked Syncthing the best out of those 3. I also considered ownCloud at the time as I was already using that for other things, but Syncthing seemed better suited for the large file library that I wanted synced in a less centralized way.
As with anything, my preferences are my personal preferences, and sometimes they aren’t for particularly good reasons. Sometimes I’ll choose to use one product over another because I just like the way the configuration works better, or its default options worked better for me than those of another project.
One thing that working with all of the customers at GigeNET has really shown me is that there are tons of different ways to set up your service for something. [quote_colored name=”” icon_quote=”no”]There’s no one right way or one right software for everyone.[/quote_colored]
I try not to get too tied to one ecosystem software-wise, but it’s easy to get comfortable with one particular set of software. This is what most end users do all the time and that’s why people get so stuck on their OS (Windows, Mac, etc) and are so afraid to jump ship to something else. I’ll redo my whole setup for something if I find a product I like better.

Q: What advice would you give to someone aspiring to become a sysadmin?

A: Google is your friend, don’t be afraid to break everything, and get involved in open source projects that interest you.
I’ve found that most of the best sysadmins I’ve worked with are self-taught, even if they have a degree. I have a degree but most of the stuff I use every day is self-taught. If you are not comfortable using Google to find answers, the best advice that I can give to you is to learn that first.
[quote_colored name=”” icon_quote=”no”]Break everything – no seriously.[/quote_colored] The best sysadmins I’ve worked with are tinkering with tons of projects at home. That usually entails setting up software you’ve never used before, and you’re going to break it a lot.
Like let’s say you followed my first piece of advice, you’re trying to set up a new service, and you’ve never set anything up like it before. So you’re following a guide that you found online. Most likely it’s not going to work. There will be something different about your setup that the guide forgot to take into account. That’s good, that’s how you learn to troubleshoot!
While optional, I would suggest getting involved in any open source communities that interest you. I’m not saying that you have to do development for them or even contribute to them in any way, but get involved.
I’ve spent a lot of hours over the years hanging out on IRC channels on places like Freenode that tailor to the open source community.
I remember when I was first learning Linux, I would hang out on IRC channels like #ubuntu on Freenode and marvel at how smart some of the people in that channel were. After awhile of experimenting and hanging out in the community, an evolution happened, where I started to marvel at how inexperienced some of the people there were.
The key is to be responsible with that realization, and if you have time, help some of those people. I always try to give back at least by answering questions when I can, because someone took the time to do that for me at one point too.
There’s a lot of knowledge out there on forums and IRC channels. So use that as a resource first, and then if you feel like it, give some help back later where you can. Even when you are helping others answer the simple questions that you are now feeling comfortable answering, you’ll run into more roadblocks and you’ll learn from that experience a lot of the time. This is a fantastic way to self-teach skills that you want to have.
Even with all my experience, there are still times that I get stuck and if I can’t find an answer on Google, I will go back to the IRC channel for the project and ask about my issue there. This has led to many fun and interesting experiences, I remember one night talking with the person who wrote the open source Nouveau driver for nVidia graphics cards on Linux. It felt like such an honor to me actually to be talking to him and he was right there in the #nouveau IRC channel on Freenode and he happened to respond to a question I asked. These kind of cool experiences have shaped me and my knowledge a lot over the years.
[quote_colored name=”” icon_quote=”no”]Basically get out there on the Internet and learn what interests you, it’s all out there.[/quote_colored] Let GigeNET lighten your workload.