What the programming is really about?

The way we learn to program causes in my opinion a small misunderstanding of what programming really is. All tutorials, most of university courses (at least these called 'programming'), and the way we approach the process of learning put focus on writing a code in a particular language. But the process of creating a piece of (usable) software is much more than just writing the code.

Whatever approach, workflow, or strategy we employ there are always some steps and parts that cannot be ommited such as design and testing. In the beginig of our carrier as a programmer we often do not do explicit design or testing steps - the programs we write (e.g. to complete programming course) are often small enough to do the design on-the-fly in our head, and refine in process, while the testing is often done in "lets check if it works" fashion. As such these steps are not completely ommited, however neglecting them this way causes our software to be only good enough to serve the particular purpose of the moment (i.e. passing the course). Extending, maintaining or even proper debugging is a pain even though the code is usually less than a few thousands lines.

But what with real software? Not toy programs of 1000 lines but real program that has let say 100 000 lines of the code, usees number of external libraries (additional 100 000 lines of code, not maintained by us directly), a number of resources like images, data files, devices (e.g. printers). A programs that are not single-threaded (often different parts do not even work on the same PC), and that compete for resources with other processes.

Extending and maintaining such an application was one of the first tasks when I started to work as a professional progammer. Unfortunately the code was written without any design, with virtually no documentation and not even trace of user requirements. To make things harder the programmer who wrote it left the company shortly before I started... The effect? Complete mess... It took me half a year to really understand how it works and fix it to the degree it could be used by the customers. 5 years, 2 releases, major rewrites, and a lot of cleanup later, the code was still simply bad, but there was hope to make it right!

Why this happened? There are several reasons most important are lack of initial requirements, understanding customer needs, lack of any design and testign routines. Plus a few missed deadlines, and a complicating factor of the customers changing their mind every second day. The code I inherited form the previous developer was simply a stack of ad-hoc solutions to the unstructured set of ideas that customers had.

To be honest I was part of the problem as well - being young and proud, I said "yes I can make it working". I was right I could - but today I know it could have been done at half of the cost and much faster if I started from scratch with the correct approach. Unfortunately, nobody prevented me from making this mistake back than as the company I was working at was not really a software development business. As such no project manager had any real experience in real-world software development.

But hey! Saying I was part of the problem I didn't mean I had no knowledge about software engineering. I had! I have taken the courses, I have read the books, and hell I even have written a few programs using the correct approach. The problem was that I had still belived that I can manage to do ad-hoc development on such a large program. Having experience only in writing programs that were 1/10 of its size, and experience in modifying large programs that were nicely written, correctly designed and well documented, I had no idea it might be that hard.

But was that really my fault? I had come across 3 universities in 3 different countries, in each case the programming courses I had to take, taught me to write a code. The engineering process was taught in separation and required me - a student - to write a small program using widely accepted approach (e.g. Rational Process). But the scale was not right, writing 1000-5000 lines of code to make a small database of customers and products makes no difference if I make some serious mistakes while building the program or not as I can recover quickly by deleting 100 lines of code and writing them again. Plus nobody told me how to estimate time and cost of the development. How are you going to earn money as a company if you have no idea what the cost will be and how much customer has to pay? Most of the graduates I interviewed over last few years for various positions in the companies I was working for were in the same position as I had been back then.

Fortunately there is more emphasis on design and testing in the current way of teaching, but I would say even more is needed. In the couple of next articles I will try to develop a small programming 101 series. I will not make a tutorial though - rather I will try to focus why we should not consider programming to be code writing (which is coding), how to avoid mistakes, and how to make ours (i.e. programmers) and our project managers life easier. I will not limit myself to write essays - instead I will be presenting various projects, techniques and examples I consider useful or because they are simply worth looking at.

Before I finish I will give you a link to a small article titled "Top 5 Project Failure Reasons, or Why My Project Fails" published on My Management Guide site. Even though I do not think it is comprehensive - it is a good starting point to understand why the project I described above was such a pain, and why it actually is a miracle it was completed to the customer satisfaction.