technical debt

Steps to Follow when Reengineering Code

Developers know that a software system will become more complex and more highly coupled over time as additional changes are made. Often this calls for refactoring or reengineering the code to make it more maintainable. Reengineering will allow you to incorporate what has been learned about how the code should have been designed. This is the kind of learning that was the original basis for the term “technical debt.”

So how should we go about reengineering code that remains vital and useful? In real life we keep applying metaphorical Band-Aids as we make changes and incorporate new technologies. This leads to design erosion. Many progressive managers now understand the debilitating nature of this erosion and how it affects quality and productivity.

code refactoring

Even if we agree that reengineering is called for, how can we plan for it? Here are four key steps to take if you have decided to reengineer your software.

1. Understand the current structure of the code. Always resist the temptation to reengineer without a clear understanding of what you have. Understand and identify the critical components and what their dependencies are. For example, if you are Java or a .NET programmer, understand the various JAR files or assemblies and how they are related to each other. For C/C++, understand the executables and libraries, as well as the code structure used to generate them. Now ask yourself: Are these the right components in my desired architecture? Sometimes you have only one component. If that is the case, ask yourself if you need to split up the component into smaller components.

Read our other blog on Reasons NOT to Refactor

2. Examine the internals of the components, particularly the larger ones and the more important ones. Look at the dependencies of the classes or files that constitute the component. Is there excessive coupling? Does this coupling make the code harder to maintain? As a team, decide what your desired architecture is. Consult senior developers. Ask the team members with different areas of expertise to validate your ideas. The testing team can be particularly helpful. A good architecture will make a huge difference in how easy and effective it is to test. You should be able to take the existing classes or files and build new components. Try various what-if architectures to arrive at the desired architecture for your existing code.

3. With the desired architecture in hand, you should now know what changes are needed and what the unwanted dependencies are. Prioritize the dependencies to fix based on your requirements. If you have areas of code that change frequently, you should think about componentizing them. Always take into account your current requirements. While reengineering has its own benefits, it is unlikely that you will stop making other improvements during this time. Any reengineering effort is likely to be in conjunction with other improvements. A good reengineering tool will allow you to perform reengineering work in conjunction with making continued enhancements to the product. Another benefit of this approach is that it will build management support for the reengineering effort.

To learn more watch our Webinar on Reengineering Legacy Code.

4. The last step is to make sure you communicate the reengineering plan to the entire team. With a prioritized scheme, make reengineering a part of continuous integration. You can create rules that prevent things from getting worse by continuously examining the changes against the desired architecture. Reengineering stories should be part of agile planning just like any other stories. Not only can you do reengineering, you can make it part of your normal development. The best reengineering exercises often minimize disruption and still allow you to migrate to a new architecture.

Vetting Software for Acquisition or Investment

Often when vetting software for acquisition or venture investment certain questions go unanswered. Typically in these situations buyers and investors focus on market share, gross revenues, projected earnings and other financial data. While focusing on capital debt, many will ignore technical debt. Some key questions are left unanswered. What is the state of the code? What is the quality of implementation? How easy will it be to fix and to add new capabilities?

Robert L. Glass talks about the “60/60” rule where he states the maintenance of software will consume 60% of your costs on average.1 These costs can escalate with software that is complex, riddled with dead code, inadequately structured, buggy, and that contains multiple implementations of the same functionality. Supplement this with a lack of documentation and developer turnover, and the system will be very costly and time consuming to maintain.2 Even when documentation is available it is typically no longer accurate as both design and code have changed. Floris and Harald in a recent study concluded that incomplete documentation is an important factor in increasing the cost of maintaining code.3

Having a firm understanding of the current state of the code will help you better understand the long term costs of maintenance and enhancements. An understanding of the architecture of the code prior to an acquisition or investment can help you make a more accurate valuation.

One of the reasons why due diligence tends to overlook code is because the process to analyze it is manual and can be costly and time consuming. For a large, complex system it could take months to truly understand where things stand from the perspective of modularity, security, performance and other key attributes.

With Lattix Architect you can understand and document the architecture in days instead of months. A DSM (Dependency Structure Matrix) will not only give you a graphical view of the source code but also show you unwanted dependencies, cycles, and other architectural defects. Lattix Architect provides important reporting metrics such as stability, complexity and coupling to benchmark and to track the code. Armed with this information you can make an informed investment decision. Knowledge of long-term maintenance and development costs can also be a negotiating factor at the time of acquisition, thereby saving upfront capital and diminishing long term costs.

Lattix can help you analyze a code base in a company that you are preparing to invest in or acquire, or if you simply want to get a better understanding of your current software portfolio. We have helped companies all over the world improve the quality of software and we can help you achieve the same results. Contact Lattix at sales@lattix.com or call 978-664-5050.

1. "Frequently Forgotten Fundamental Facts about Software Engineering" by Robert L. Glass, (an article in IEEE Software May/June 2001)
2. “On the Relationship between Software Complexity and Maintenance Costs” Edward E. Ogheneovo - Department of Computer Science, University of Port Harcourt, September 2014
3. “How to save on software maintenance costs” Floris P, Vogt Harald H., Omnext white paper, SOURCE 2 VALUE, 2010

Just Enough Anticipation

How much "architecture" is good for agile development? How should you think about the future implications of design as you write code to meet immediate requirements? A recent article in CrossTalk, tackles this subject head on. The authors - Nanette Brown, Robert Nord and Ipek Ozkaya are from Software Engineering Institute (SEI) and well known for their prior contributions to the study of software architecture.

In their own words: Our mantra for Architectural Agility is “informed anticipation.” The architecture should not over-anticipate emergent needs, delaying delivery of user value and risking development of overly complex and unneeded architectural constructs. At the same time, it should not under-anticipate future needs, risking feature development in the absence of architectural guidance and support. Architectural Agility requires “just enough” anticipation.

According to them, tools such as dependency management, real options analysis and technical debt management can help you strike the right balance. Check out this thought provoking article: Enabling Agility Through Architecture.

Modularity as a Portfolio of Options

I have been exploring the use of financial analogies with regard to programming and design. Ward Cunningham's Technical Debt metaphor has become well known. Prior to writing this blog entry, I looked a little deeper into Ward's metaphor and discovered that it has been interpreted and extended in multiple ways. Since this is my view of the different interpretations, I recommend that you go to the actual source to arrive at your own conclusion.

First let's examine how Ward used it originally. He used the metaphor to explain the need for going back and improving the code to incorporate what you learn after an actual implementation. I believe that there are a few good reasons for this suggestion:

  • We often don't know the best way of implementing something until we actually implement it.
  • The customers learn what they really want only after they have seen an implementation.

So for Ward, technical debt is a good thing, perhaps, even a necessary thing. An implementation, even a suboptimal one, allows you to learn what to implement and how to implement it. This is pretty much the original raison-d’être for Agile.

Uncle Bob applied the metaphor for an example of an implementation that is known to be sub-optimal in the long term but allows you to get things done in the short term. While Ward considers the debt as necessary for understanding the selection and design of the right system, Bob used the metaphor for a worthwhile engineering trade-off. Steve McConnell and Martin Fowler would also include poor coding or “quick and dirty” implementations as technical debt.

If the value of the metaphor is for explaining to others why improperly conceived code will extract a price during future development, just as a debt extracts an interest in future years, then I think that the metaphor works for all these different situations.

But now on to what this article is all about. It is about another metaphor - a metaphor that also goes quite far, I might add. This metaphor comes from Carliss Baldwin and Kim Clark, from their book, Design Rules: The Power of Modularity. It too deals with how the current design of a system impacts the cost and the value that can be realized from it in future.

According to them, a design yields a portfolio of options. For a modular system, the options are the opportunities to improve the design in a modular or piece meal fashion. A monolithic design, by contrast, gives you a single option. You have to redesign the entire system to improve it.

Baldwin and Clark point to a well known theorem in finance - it is more valuable to hold a basket of options for many different securities than it is to hold a single option on the entire portfolio. So it is with system design. A monolithic design gives you a single design “option” to improve the system while a modular system gives you multiple “options” to improve the modules of the system.

Consider the example of a PC. It is a highly modular. When I bought my laptop, it had an 80 GB disk. Later, I went out and bought a 300 GB disk and all I had to do was swap my disk with the new one. In the intervening period since I bought my laptop, the disk manufacturers were able to improve the disk design so that the same form factor gave me a bigger and faster disk. I was similarly able to upgrade my laptop with additional memory. Each of these modularity “options” represents an axis for further improvement that does not force the redesign of the entire system. The ease of improving the design of a modular system confers dramatic benefits to the customers of that system. This is why modular designs are so valuable.

Of course, it is important to keep in mind that the value of modularity exists only in the context of requirements. Because we need larger disks to store ever increasing amounts of data, it becomes valuable to improve the density of disks. In other words the “option” is only valuable because the value of the underlying “security” could increase. Just because you create a module it doesn’t mean that you have suddenly added value – it must help meet requirements that somebody actually needs or wants.