Technical Debt

I first came across the concept of Technical Debt whilst reading a recent copy of the Microsoft Developers Journal called the MSDN Magazine. It struck a chord, as it's a concept that all software developers are familiar with.

In every software system, there are areas of the code that are difficult to change, that strike fear into us when we are required to change them. Those areas of the code that make you think "Oh please no". They are difficult to modify or extend, and thus working in those areas are prone to making the system less stable. They will also generally require a greater degree of regression testing to ensure that you have not inadvertently modified other areas of the system.

The phrase was initially coined by Ward Cunningham, and uses the metaphor of finance. There are certain kinds of debt that are cheap to repay, where the interest rates are low and repayments incur little cost. Then there are those kinds of debt that are difficult to repay as the repayments are high. A credit card would be a good example of the latter, where their high interest rates make them an option that should only be considered as a last resort.

The major downside of Technical Debt is that is has an associated cost. Being difficult to modify will lead to longer development lead times. It will require additional testing, particularly regression testing. All of this extra time and effort involves greater cost.

It also ensures that the software system in which the Technical Debt finds itself will be less likely to be able to respond to customer changes, or fluctuations in demand or functionality. If there is a swing in customer demand, and the system needs to change to be able to respond to this swing, it will require far greater resources to make those changes in a software system which has a higher degree of Technical Debt.

I remember whilst working as a Senior Software Engineer at the Kettering based software house Pegasus, one of my colleagues telling me how he disliked modifying certain areas of the underlying Framework code, as it was like "playing Kerplunk with the code". What he was referring to of course, was that the code was so interlocked and tightly integrated, that making a change in one area of the code, would often result in changes being made inadvertently in other areas of the code. This would lead to longer development / testing cycles, and would be fraught with frustration.

I have already discussed in previous articles how to prevent Technical Debt through the use of Design and Design Patterns. I won't bother covering how to repay back already existing Technical Debt, as this too has been covered in an excellent MSDN Magazine article.

Technical Debt is a simple concept that every software developer both understands and can sympathise with. If left unchecked, it can lead to significant cost implications, so it is worthwhile investigating and taking the time to remove it. There is a strong financial incentive to make the case for removing Technical Debt, and this should be proposed as a project as early as is convenient.

Why design is critical to software development

A good software engineer should always design
I recently had a discussion about the merits of design within the field of software development. My protagonist was of the opinion that design is not something that should be taught in the early stages of becoming a software developer, and that design was largely a matter of common sense anyway. Their opinion of design patterns was that they were too time consuming to be of use in commercial applications. I will address each of these points throughout the course of this discussion.

As a professional software engineer with over a decades worth of commercial experience, and having worked in both good and bad development environments, I know the benefits that come with good design.

I don't intend to describe each of the various design methodologies or design patterns as there are numerous books and articles already available on such subjects. I intend instead to describe why design is a crucial part of the software development life cycle.

Software is an engineering discipline
I always like to use the analogy of a software engineer and civil engineer. Both should employ engineering concepts to achieve their objectives. What this means is that when designing a software application, you should borrow engineering elements from the field of civil engineering.

Here is a definition of software engineering taken from Wikipedia:

"Software engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software".

When designing a suspension bridge to span a mile wide river, would you really want to just proceed without giving any thought to how you intended to build it. Just turn up one Monday morning, and start bolting steel girders together. The obvious answer is of course not, but this same level of diligence and engineering does not always seem to apply to software development, even it would seem from some of those working within the industry. It would be classed as utterly negligent to build a suspension bridge without having a design in place first.

The earlier you are taught design, the better. Imagine turning up to a course on car building, where the lecturer said "This week I'd like you all to begin building your cars please, and next week I'll show you how to design them". I'm sure you would think there was something wrong with the way the course was being taught.

Before you build something, anything, you design it first.

Software design is not a trivial process
Whilst some elements of software design may be common sense, that does not necessarily imply that it is all common sense by extrapolation. In fact, it is fair to say that the design process can be difficult and demanding. It can take years to develop the experience, knowledge and skills to become expert at it.

Design constraints including extensibility, reusability, tolerance, coupling, cohesion and reliability to name a few, must all be carefully considered. There will also be many trade offs and compromises that must be considered as part of the design. To say that all of this is simply common sense is nonsense.

The design process also manages the complexity that is inherent when building a fully fledged software application. With so many considerations to take into account, the human brain can quickly become overwhelmed with information. This needs to be broken down into manageable chunks that a designer can understand and work with.

Design is the key to communicating your intent
If you don't have a design, how do you and your team know how you are going to build the solution? Design takes time to get right, and can be fraught with problems. I spent several years as an Analyst Programmer, where part of my time would be spent visiting customers to document their requirements. I would then work this into a document explaining as clearly as possible how I had interpreted their requirements. This would also include using use-case diagrams using the formal notation of Unified Modeling Language (UML).

I never got this right the first time. I would send this to the customer for their input, and they would invariably point out the omissions and inaccuracies, for me to then revise the requirements document. After several iterations, you would arrive at a set of requirements that the customer would be happy with (for now, but that's a whole other story).

A similar process can and should be employed in the design process, taking the requirements document, and iteratively shaping this into a working design. The software design should form the key communication tool amongst the software team.

What do you mean you don't have time?
I have heard many excuses of why the design process is often skipped, the most common one being not having time. My reply to this is "you don't have time NOT to." Seriously, spending time getting the design right, just like spending time getting the requirements right, will always save you much more time over the longer term than it will cost you in the short term.

To rectify a design fault can be costly, but to rectify a source code fault is costlier still. The later in the software development lifecycle you find a defect, the costlier it will be to fix. So one way of looking at the need for design is to reduce the costs of defects that will be found in the implementation and testing phases (finding a defect in the testing phase is the costliest of all).

By spending time designing, you are actively trying to reduce the number of defects that will be found in the later stages of the software development lifecycle. You are also mitigating any risks by actively seeking out a working solution, rather than proceeding regardless, only to find later on that something is not possible, or must be compromised.

Design can reduce the risks and costs to your software project.

Benefits from adopting software design:
There are many benefits to incorporating design into your software development lifecycle. Here are a few examples:

• Communication - the proposed solution is visible to the team and all project stake holders, and so everyone has a clear idea of what is required.


• You can find problems with the solution before you start the implementation stage, where re-development will be far more costly.


• Maintaining the application will be far easier when you have a series of design documents to refer to. This makes handing the maintenance of the application over to another team or third party much easier.

Conclusion
The design process is crucial to the success of any software development project. It forms the central communication document for the development team. It reduces the risks and costs to the project. It helps to manage the complexity that is inherent to the development lifecycle.