Software-based systems are arguably the most complex man-made constructions in existence. What is more, business is increasingly dependent on such systems for smooth running of operations. There are few modern companies that operate without software, and certainly none of any size. With this vast complexity of implementation, the only way that software design can be structured to make it manageable for humans is through abstraction - the process of ignoring unnecessary detail at various different levels. This is common across all engineering disciplines when developing something of significant magnitude. By Jonathan P. Bowen of London South Bank University.
Traditionally, ways of abstracting software early in a design are rather informal. Obviously a written description in natural language is possible. This may typically be accompanied by diagrams as presentation aids. Such diagrams may be supported by tools and standardised. For example, UML (Unified Modelling Language) is now used as a standard set of approaches for handling and presenting abstraction from various viewpoints in a diagrammatic manner.
However, a problem with natural language and diagrams is that the meaning can be unclear. The customer may assume one interpretation and the developer another. It is only when a more formalised version of the system is produced (e.g., the software code itself) that some errors become apparent. This then requires an expensive return to a higher level of abstraction to address the problem or, worse, a fix that may have unexpected consequences for the overall operation of the software.
All software artefacts become formalised at the implementation stage in that the exact meaning can (potentially) be determined in a rigorous and precise manner. This is because all computers can be considered as a mathematical machine, a property identified by the 20th century British computer pioneer Alan Turing that is still very important to this day. Software specifications and designs do not necessarily share this property and indeed typically do not in most industrial projects.
However, there is no reason why this should not be the case, apart from the additional effort required. This is frequently cited as the reason why such an approach is normally not taken in practice. Managers prefer not to commit additional effort early in a project if possible.
This can be partly because payment is not available from customers until later in a project, so cash flow can be an issue. But such investment can pay significant dividends later on. A NASA study has found that additional effort at the requirements stage typically reduces cost overruns in projects, whereas lack of investment early on has the opposite effect.
Approaches that encourage more rigour at the early stages of software projects are collectively known as ‘formal methods’. There are many such methods available and it can be confusing to decide which approach to adopt on a particular project. Availability of staff with prior knowledge of a specific formal method can be important since training is a significant issue. The one-off cost of learning to use a particular formal method is high, but once done, this knowledge can be applied in many subsequent projects.
The British firm Praxis High Integrity Systems, based in Bath, has many staff trained in the use of the Z notation, a well-established and generally applicable formal approach. This means that Praxis typically uses formal methods on projects even if the client does not specify this as a requirement, since it can produce higher-quality software, ultimately at lower cost – because fewer changes are needed later due to misunderstood requirements.
Formal methods are not widely used in software development in general. However, they have achieved niche use in high integrity systems, where safety or security is paramount, or the cost of failure would be huge (i.e., in business-critical applications).
The impact of using formal methods in software development is a significant and often misunderstood issue. Most software projects start with a set of informal requirements agreed with the customer, ideally in advance, although requirements are notorious for changing during the lifetime of a software project.
In the table (Fig.1), by way of example, a hypothetical project where the initial requirements are only 25 lines in length is shown. These requirements are typically the result of initial discussions between a supplier and a customer. It may be desirable to formalise even this high level of abstraction, for example, using a domain-specific formalism that allows a concise description of the particular problem being tackled.
_____________________________________________________
Fig.1 Levels of complexity
25 lines of requirements
250 lines of specification
2,500 lines of design description
25,000 lines of high-level program code
250,000 machine instructions of object code
2,500,000 transistors for a computer in hardware
_________________________________________________________
The contractor can take these requirements and produce a specification document, detailing what the system is intended to do. This is still typically informal, but alternatively it is possible to formalise the system for the first time at this level of abstraction. This could be, for example, 250 lines long (i.e., an order of magnitude larger that the original requirements). Unless the various levels in a development process introduce significant additional complexity and information, there is little point in including such a step.
Next it is normal to describe the design (an architectural description) of how the desired software is to operate in more detail, including algorithms, data structures, etc. Again, much more information is introduced, making the design description 2,500 lines long, for example.
Typically, UML is used at this level of abstraction. More formally, a design tool could be used to develop a high-level specification to a lower-level algorithmic description, ideally in the form of a compilable programming language. Examples of such tools include Perfect Developer, using an object-oriented approach, and the RODIN tool, based on the B-Method. The latter is being developed by a collaborative European project of industrial and academic partners. The tool pays particular attention to a good user interface and the management of changes in the specification, which is a very important aspect in real-world projects.
The program produced from the design stage, whether created formally or informally, could be an order of magnitude larger than the design documentation (i.e., 25,000 lines of code in our running example). This is typically the first time that a fully formal piece of text is produced on most industrial software projects. In this sense, all software development involves formal methods, but normally the formality is introduced at the last possible stage, as an executable high-level program.
By this time, this executable program is many times larger than its counterparts at the requirements/specification stages. Thus, if only because of scale, it is much more difficult to analyse and manipulate. In addition, most programming languages are not designed for reasoning, although industrially-developed programs now include assertions for testing purposes, originally developed by C.A.R. Hoare and others in the 1960s for program proof. Indeed, a significant percentage of code produced by companies such as Microsoft consists of assertion code.
Once an executable program is achieved, the process of generating lower-level descriptions can be automated with fair reliably, using a compiler to produce object code suitable for direct execution by a processor. This may have more instructions within than the high-level program equivalent (i.e., 250,000 machine instructions in our example). This is implemented in hardware of yet more complexity, perhaps 2,500,000 CMOS transistors, for example.
Formal methods can be introduced and used at any of these levels of abstraction. However, the impact is likely to be much more cost-effective at the higher levels, such as those for requirements or specification. At these stages, the description is still relatively small and can be changed easily at little cost. At each development stage, the cost of correcting errors becomes much more expensive.
It is highly desirable to correct errors early on before proceeding to further stages of development. However, this can easily give the appearance of delaying a project, because it can seem to a naïve observer (such as a manager inexperienced in the use of formal methods) that there has been little progress with respect to the generation of executable code until relatively late in a project which makes use of formality at the initial more abstract stages.
Unfortunately, not all software developers are experts in abstraction. Indeed, this seems to be a skill that is innate in some people and much less well-developed in others. Those with this ability tend to produce better software more quickly. An article by Jeff Kramer entitled Is Abstract the Key to Computing? in the Communications of the ACM (April 2007) highlights the problem. Teaching abstraction is difficult and employers of graduates in the software industry would do well to seek those with an aptitude for this important ability.
Jonathan P. Bowen is Emeritus Professor at London South Bank University. He is also on the faculty of BCIM, and Visiting Professor at King's College London. He is a director of Museophile, Ltd., which aims to help museums, especially online. Email: jonathan.bowen[at]lsbu.ac.uk
The articles published here in the Thinking CEO are internet updates of the latest management knowledge and practice, which have been commissioned by Sovereign Publications for their bi-annual magazine, CEO Today, and will appear later in the first 2007 issue of this publication. To contact Sovereign and CEO Today, go to:
http://www.sovereign-publications.com/ceo-art.htm
