When is it time for a senior sysadmin?

In the quest for the "perfect" startup to join, I have my own personal guidelines as to company size and growth. However, I also tend to ask questions to determine if it's too early or too late for me (as a system administrator) to be of adequate help.

I'm not just a porridge-swilling Goldilocks when it comes to this kind of timing. If it's too early[1], I'm going to get bored, while the company wastes its money, which isn't good for anyone. Too late, and I end up being incapable of overcoming legacy hurdles, which is a source of frustration and appearance of ineffectiveness, again not being good for anyone.




Growth

The first thing I look at is growth, since that's the single most reliable sign that it may be "too early." Merely modest growth can also mean a great challenge for someone who is by all accounts still a senior sysadmin, but not for me, which is why I probe that early.

For a startup with the expectation of "hockey stick" style growth, I would say the right time is anywhere on the elbow part (of greatest slope change). The nearly-horizontal part means it's too early, since that can last an indeterminate amount of time and such minimal[2]  growth can be handled by developers sharing the load of administration.

I look for 10% monthly growth or doubling yearly as a minimum. Any metric that can be credibly linked to infrastructure works, including bandwidth, users, revenue, servers, even employees. I have yet to run into the need for having a maximum. Does anyone have a suggestion of a growth rate that's clearly up the handle of the hockey stick? Factor of 10 yearly?

Employees

Another metric I use is number of employees, or, more specifically, number of technical employees. The "too late" case has an easy rule of thumb: everyone technical needs sit in the same room and still be able to communicate effectively with each other. My experience is that this is a common early startup model. Once people have walls (even cube walls) and doors separating them, there's just enough of an "us vs. them" mentality that a sysadmin can no longer absorb enough of everything that's going on to effectively influence how things are done in the future[3].

I'm not sure there's a danger of there being a "too early" case, but I'd be hard pressed to recommendsysadmin being ones first or second hire.


Number of Servers 

A common, though, to my mind, less significant, metric is number of servers. The reason I consider it of secondary importance is that doesn't translate well to overall environment complexity. Put another way, the existing number of servers doesn't translate well to the eventual number of servers once a sysadmin.

Still, if you can run everything on one or two servers, it's probably too early. If you have a couple hundred and you don't already have someone dedicated to thinking about them, it's too late.

Server/Services Spending

More significant than the number of servers is how much is being spent on the hardware (if applicable), hosting, and services, such as ISPs and CDNs. My sweet spot is that this needs to be about twice the salary of a sysadmin, since I can often cut those expenditures in half[4]. Less than a sysadmin's salary and it's too early. More than 5 times a sysadmin's salary and it's too late, though, like with growth, I have yet to see this be an issue in the real world.



[1] Granted, it may only be too early for me but not for a junior sysadmin. That's a philosophical question for another post. I've found, however, that most startups don't want to spend the time and money to eventually hire two people rather than waiting and getting just one.

[2] It's important to remember to normalize against technological progress. Even I/O performance progresses linearly, even if it doesn't follow the geometric progression of Moore's Law.

[3] Including influencing development process and tools, if not providing the outright. I've heard this method called "DevOps," but I just consider it to be good startup system administration.

[4] Easily justifying my own salary, if needed, but, more importantly, revealing the negotiation over $10k one way or the other seem the silly waste of time that it is.