Requirements I

As discussed in my introductory posts, requirements engineering is one of the most important factors of projects' success (or failure). If we take a look at The Standish Group CHAOS reports (well you have to have $99 to spend for a recent copy; some old reports are available for free "for academic purposes") we will see, that over 44% of failures occur due to lack of proper requirements management (i.e. lack of user input, incomplete requirements, unrealistic expectations, changing specification etc.). Another  26% are related problems like: lack of proper planning, lack of resources (how can you plan activities and manage your resources, time, and money if you don't know what really needs to be done? How do you create budget and set up price the customer is going to pay?). Surprisingly almost 8% of project are terminated because the customer does not need it any longer - a problem also related to requirements specification, as if they were performed properly in the beginning, many such instances would be detected much earlier.

But what requirements really are? The short answer is that requirements define what users, customers, suppliers, and business routines (lets call them stakeholders for now) need from the system, so it is useful and helpful for them. However, be careful as requirements from different stakeholders may be contradicting, ambiguous, or incomplete!

Why do we need requirements in a first place? As mentioned before - we need them for planning and risk management. We also need them to do proper acceptance testing, to manage trade-offs to be made (and justify them!) and to manage change within the organization that is going to use the deployed system.

We cannot think solely about the software when we start to collect our requirements. Lets imagine, that we are building a very complicated web application. The application is going to deliver a vast amount of data to some other systems around the globe. The application alone is not able to perform the task - if the server and uplink are to slow, than even the best application will not be able to send the data as quickly as it needs to be. Similarly if the third party systems are expecting different format than we send they will not be able to use the data. As we see the correct result - requirement - can be achieved only if we think about the whole system - software, hardware, and human along with interactions between them.

Very important thing to remember is, that requirements management is not a single step in development (like often shown by waterfall model), but rather they are constantly involved in all phases of the project: planning, development, testing etc. Requirements also help to track the progress of the project - not only by developers, but also management staff. They help understanding (at fairly high-level) what is developed and in many cases might help reuse of components across projects, coordinate activities, or share resources.

Another thing to consider is that business requirements (usually what our stakeholders talk about) is not equal to engineering requirements (what we need to develop a system). In general we can say that there are at least 3 layers of requirements analysis. First stakeholders view (their expectations and objectives) are translated into set of stakeholders requirements - defining what stakeholders want to achieve by having the system. In this step we are focusing on the problem domain, usually trying to avoid any particular solution. In next step we translate these requirements into set of system requirements - defining (at high-level, without specific details) how the system will meet the stakeholders needs. Than we design the system effectively translating the system requirements into architecture of the system. At this step we define how this specific design will meet stakeholders needs. The 2 latter steps are expressed in the solution domain.

For example lets consider a case, when a stakeholder is in need of a personal safety system for a excursion vessel. They might express their need by saying that they need means of ensuring personal safety should he vessel sunk, yet the item their customers have to wear while boarding should not impair their comfort and ability to move freely. At the system requirements level we will consider all possible solutions such as vests, life rings, or Personal Anti-Gravity Module (given it was already invented) to pick one solution that will best suit the stakeholder needs. We will discuss how this solution is going to fulfil stakeholder needs, but we will not go to much into design details. At the design level we will think about this specific solution, and design an instance of this solution - like a specific dimensions, material, shape, and colour of the vest (or life ring, or PAGM).

Moreover in many cases we need to translate high level requirements (goals or expectations) in a set of low level requirements. The relations are often quite complicated (many-to-many), hence understanding of relationships between different layers (it is called traceability) of information is needed.

It is often the case, that stakeholders not only specify their needs in terms of a particular solution - usually one of more IT literal person has read something about this new cool thing called 'peer-to-peer' and they want it even though they do not know what it means, but also focus on very low level issues such as colour of buttons and their shape, rather than clearly state their needs and goals to achieve. In such case the requirements engineering should lead to justification of particular demands (maybe simple client-server approach would suffice?), or we will not be able to find the most optimal solution. Also a clear distinction of solution and problem domains needs to be made. Otherwise the discussion will be taken over by developers, as the only description will be really done in solution domain, resulting in lack of deep understanding of the goals to be achieved and functionality to include in the system.

In a summary I will give an example of things going slightly wrong because of unclear specification. A few years ago I was leading development of an auction system. The initial requirement was to have possibility to bid on items until the given deadline. After some time the requirement changed, so that if the bidding was done within last 30 seconds, than the deadline is extended by additional 30 seconds (effectively the more people bid, the longer the auction). Connected with that was demand of having a real time clock displayed on the bidding page showing time to end of the auction. Everything was OK at this point. However later the requirement changed again to have possibility to have 30, 15 and 5 seconds extension times - and here the problem started to be visible. The typical round-trip time to server the customer had was over 600ms, moreover the load of the server was enormous due to number of bidders (+ overhead from the clock update). The customer wanted the clock to be precise (means he had 4 laptops on one desk and he wanted all the clocks to be counting EXACTLY the same with millisecond agreement). A requirement impossible to satisfy - simply because such synchronization is not possible to achieve (see article Probabilistic Clock Synchronization by F. Cristian if you are interested). The best we could do (along with mathematical proof!) was about 200ms - a small, but visible lag between the terminals. So we had everything - unrealistic expectations, changing requirements, incomplete requirements, and in the initial part of the project lack of customer input... The project was deployed successfully (with small delay), but the customer went out of business soon afterwards due to complete lack of marketing (and lack of budget as they have not done any budget predictions...).

Summarising: this post was again pretty theoretical. The next will be discussing a practical ability to write down requirements. We will also discuss some approaches to modelling, as it is often very useful in supporting requirements management (and many other activities as well).