Depending on whom you ask, the very nature of system administration varies. Is it art? Black art? A profession advanced through the apprenticeship model? Are we smithies? Many would argue, especially the virus fighter or self-healing system designer, that system administration is a science, with natural rules and reactions.

Others might suggest, and often do, that we're somehow engineers. I'd argue that most of us are closer to locomotive engineers than to the trained and licensed engineering professional, one who uses repeatable processes, measurement, and design; the hallmarks of the engineering profession.

In the next few articles, I'll be walking the border between the province of the engineer and the shire of the common system administrator. I'll be using some textbook definitions of engineering terms and applying them to the things we do in our daily work. I do this not as an indictment of my very own profession, but in hope of peeling back the layers of myth and guesswork so prevalent in designing and running large computer systems. We've made tremendous strides in some areas, so all is not lost.

Many of us know our systems and applications deeply, and like our children, we know when they'll misbehave. To me, though, that's not a repeatable or easily teachable relationship or one that serves us well in the engineering or business worlds, where certainties such as total cost of ownership and project completion dates are expected.

Why do this? I'm interested in advancing our cause and strengthening the differentiation in unskilled vs. skilled, between the carpenter and the guy that built Taipei 101. Rather than simply stoking the fires and twisting the throttle, I'd like to see us closer to being CAPCOM, coolly wearing the vest and shepherding the mission (our system) that was scoped well, designed correctly, and responds in known and predictable ways.

We can improve the use of tools and measurement and make decisions based on hypotheses confirmed with data, rather than gut-feelings. The outcome is bound to be better systems, ones certainly better in cost of ownership and efficiency. If we've designed systems correctly, with a nod to modularity and standards, they'll scale better and our time-to-market will improve. Much of this is self-apparent to many of you, I'm sure. It hasn't reached everyone yet.

By way of definition, I'm using the terms "system" and "computer system" to describe a set of computers, processes, and interconnections, typically in a large installation supporting large corporations or government agencies. These are places where volume and traffic require true efficiency and scalability. I'm not describing the world of the Electrical Engineer, designing the actual boards or integrated circuits, nor are we hoping to apply much thought to setting up a print server for the twenty-five person company. We're not interested in the heat generated by the CPU or the hard disk, except in the most macro way, i.e. "how do we minimize cost, be that in energy consumption or HVAC, with our implementation?"

Let's start off with the engineering design process. I'm using an engineering textbook, Engineering: An Introduction to a Creative Profession [Beakley, Evans, Keats, 1986]. The specific naming of the stages is not so important and perhaps will vary according to the problem or system at hand, but the process itself lends us structure and provides a known path to progress from Need to Project Plan.

Stages of Engineering Design

1. Identification of the problem
2. Analysis
3. Transformation
4. Idea Development
5. Modeling
6. Information Gathering
7. Experimentation
8. Synthesis
9. Evaluation and testing
10. Presentation of the solution

Phew! That's quite a list of things to do before you build that new mail server, right? Well, that mail service is the thing that needs the process, not your lowly server. The ground-up design of the system, following industry and de facto standards is the time and the place for the steps shown above. Let's delve into these a bit, shall we?

Identification

What is it that is needed? Seems simple, right? Requirements gathering and scoping are two of the major gotchas in this business. The marketing and business people come bearing grand, shiny ideas, marveling at their own cleverness, only to be ground down by the reality, the lack of support from legacy backend systems, and the endless questions from the 'engineers', who are now seen as a roadblock to be bypassed.

This is where we must perservere; to dig, to not take things at face value, to ask the same questions in different ways. You must not only ask what is needed, but what is not needed. In helping the customer to fully identify the problem, we should consider some of the following areas and constraints in this first phase.

• Can the purpose of the system be reduced to a single sentence or paragraph? Have we clarified to that level?
• What is the scope of the problem? How many users, customers, data sets, or records are affected?
• What are the interconnections, data flows, and processes that will feed our new system or allow it to feed other systems?
• Can we identify, even in rough terms, the availability of funding and management commitment to the project? Is the system part of our charter or corporate mission?
• What are our legal and ethical constraints for this system, its data, and its users?
• What is the expected outcome of this design exercise? Are we simply studying the feasability of solving the problem or actually moving to do that work?

Write these things down. The use of standardized forms or questions can help guide and document the discussions, but don't rely on them alone. Repeat back your understanding of the problem and constraints to the customer, letting him hear what you understand to be the problem. One other useful tactic in discussions of upgrading or replacing systems can be a review of the limitations of the old system.

With proper problem identification, you'll have formed an agreement with all concerned over what is actually to be fixed, designed, or upgraded, thereby reducing feature-creep or mismatched expectations later in the project, when refactoring or modification will be expensive and disruptive. Don't skimp at this stage.

Next time, we'll jump into the analysis stage for some thoughts on problem decomposition and application of known data to the problem that we've identified.

If you've got suggestions or maybe comments on how I've gotten it all wrong, hit the Comments link below and let's hear it.

Technorati Tags: system administration

Tags: on technorati, delicious, netscape, google