software architecture

Software Architecture and GDPR Compliance

The General Data Protection Regulation (GDPR) is an EU regulation on privacy protection that goes into effect in May 2018. GDPR applies not only to EU companies that process personal data on EU residents but also to companies not located in the EU. As Article 3 states, it is “applied to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union.”

Software architecture is an important part of GDPR compliance. An architectural model of the software gives you a complete view of everything connected to the personal data in your system. The GDPR defines personal data as any information that has the potential, alone or paired with other information, to identify a person. You need to preserve the identity of an individual across different names and properties and be able to trace them across the system and disparate data points as stated in Article 30. You have to record what you do with personal data and define which applications use it.

The Automated Decision Making section of the GDPR states that any system which undertakes automated individual decision-making, including profiling (Article 22), is now contestable by law. This includes automation components such as calculation engines, scoring systems, or other processing of personal data. You need to be able to trace the personal information through these systems and demonstrate compliance. Article 5 states “the controller shall be responsible for, and be able to demonstrate compliance…”

Compliance Steps for GDPR

As part of ensuring compliance for GDPR, you will need a good overview of the personal data involved.

  1. Identity all data that the GDPR considers personal data. Lattix Architect will give you this information with its member level expansion feature that allows you to see all of the variables associated with personal data.

    software architecture (see our video on Member Level Expansion)

  2. Once you have identified the personal data, you need to analyze its use. Lattix Architect understands all of the dependencies in your software system, so it will know all of the dependencies on the personal data. Now you will be able to model the data flow and show which applications, processes, etc. use the personal data.
  3. Once you have modeled the data flow, you will be able to demonstrate compliance with GDPR by using the Impact Analysis Report in Lattix Architect. This report tells you all the dependencies on selected elements (in this case variables) and can be exported to Excel, csv, or XML formats.

    software architecture (see our video on Impact Analysis)

GDPR compliance is something you need to regularly revisit. You must go through the above steps frequently to ensure you remain compliant. This becomes part of your governance framework.

Summary

Non-compliance with GDPR can result in large fines. Penalties,as outlined in Article 38, include “fines up to 20,000,000 EUR or in the case of an undertaking, up to 4% of the total worldwide annual turnover.” There is personal damage that can be claimed by any individuals who are the data subjects and there is personal liability for directors and senior managers. This all makes it worthwhile for organizations to take these risks seriously.

Motivation for Software Architecture Refactoring

Refactoring is commonly applied to code, but refactoring can also be applied to other development artifacts like databases, UML models, and software architecture. Refactoring software architecture is particularly relevant because during development the architecture is constantly changing (sometimes for the worse; see our blog post on Architectural Erosion) and expanding. Software architecture refactoring should happen regularly during the development cycle.

We have talked in the past on how to perform architectural refactoring (see our blog post What is Architectural Refactoring?). In this blog post we talk about why you should.

Why refactor software architecture?

The ongoing success of a project is based in large part on the software architecture. Software architecture directly influences system qualities like modifiability, performance, security, availability, and reliability. If the architecture is poor, no amount of tuning or implementation tricks after the initial development will help significantly improve system qualities. You need to evaluate and refactor your software architecture early to know whether it will meet your requirements.

Software architecture evaluation and refactoring should be a standard activity in any development process because it is a way to reduce risk, it is relatively inexpensive, and it pays for itself in the reduction of costly errors or schedule delays. Architecture also influences things like schedules and budgets, performance goals, team structure, documentation, and testing and maintenance. As software teams grow and/or become more distributed, understanding software architecture becomes even more vital. If everyone on the team does not have a clear understanding of the architecture (what components depend on other components, etc.), defects start to develop in the code.

For example, if you were building a house, you would carefully examine and follow the blueprints before and during construction and make changes to them as new requirements are introduced. In construction, this extra time is worth it because it’s better and cheaper to find out the homeowner wanted an extra bathroom during design or construction than on moving day! The same is true for software development.

What is software architecture?

To properly refactor a software architecture, you need to understand what information is relevant. Software architecture is the structure of a system. It is made up of software components, their properties, and their dependencies. The architecture defines the modules, objects, processes, subsystems, and relationships (calls, uses, instantiations, depends on, etc.). Architecture defines what’s in a system and provides all the information you need to know how the system will meet its requirements. Software architecture fills the gap between requirements and design. To refactor the architecture, it has to be understandable and easily visible using Dependency Structure Matrix (DSM) and Conceptual Architecture Diagram (CAD) views.


software architecture

Finally, the architecture creates the requirements for the low-level designs.

When should you refactor software architecture?

In a typical project, you design and think about the architecture only at the beginning. But, as stated earlier, architecture refactoring can and should be applied at all stages of software development. As an example, in agile development architecture evaluation and refactoring should happen once per sprint.

Architecture refactoring is particularly helpful after implementation has been completed. This might happen when an organization inherits a legacy system or if you are put in charge of an existing application. Understanding and refactoring the architecture of a legacy system is useful because it gives you a complete view of the system and answers the question of whether the system can meet the requirements in terms of performance, security, quality, and maintainability.

What are the goals of software architecture refactoring?

“If you don’t know where you are going - any road will get you there” - Cheshire Cat

If you don’t know what your goals are or if the goals are too vague (“the system shall be highly modifiable,” “the system shall be secure from unauthorized break-in,” “the system shall exhibit acceptable performance”), then there can be a misunderstanding of what needs to be done during refactoring or when refactoring has been completed. The point is that system attributes are not absolute quantities, but exist in the context of specific goals.

Not all system attributes can be improved with software architecture refactoring. Usability is a good example. This has more to do with the user interface than the underlying architecture. Although if the user interface has its own module with limited dependencies it can be easily swapped out for a different interface (i.e. using the web instead of desktop GUI).

System attributes that are determined by the architecture and can be improved with refactoring include:

  • Performance - This is how responsive the system is in certain workload conditions (as specified by the end users) or how many events it can process during a certain period of time.
  • Reliability or availability - This is the ability to keep the system up and running over time. The system needs to recover gracefully from failures or unexpected behavior.
  • Security - This is the ability to resist or defeat unauthorized usage and/or denial of service attacks while still providing the correct service to legitimate users.
  • Modifiability - This is how quickly new features and updates can be made to the system based on changing requirements or bugs found in the field or through testing.

Summary

The benefit of software architecture refactoring is uncovering problems earlier in the development cycle when they are cheaper and easier to fix. It produces a better architecture that helps with future development and maintainability. An iterative and consistent architectural refactoring process increases everyone’s confidence in the architecture and in the system as a whole.

Software architecture refactoring gives everyone a better understanding of the architecture. This can then easily be communicated to all interested parties including product management, other developers, QA, etc. Lattix Architect is a great companion for architectural refactoring and evaluation as it makes the visualization of the architecture easier and allows for quick what-if analysis of the architecture.

The Smell of Rotting Software

Jack Reeve introduced the concept that source code is the design and programming is about designing software.1 As software grows, the design, or architecture, tends to grow large and complex. This is because software architecture is constantly evolving, making software maintenance difficult and error-prone. In this article, we will talk about symptoms of bad architecture and how to fix them.

Poor Software Architecture

According to Robert Martin2, there are seven symptoms of poor architecture.

  1. Rigidity: this means the system is hard to change. Every change forces other changes to be made. The more modules that must be changed, the more rigid the architecture. This slows down development as changes take longer than expected because the impact of a change can not be forecast (impact analysis can help). System stability and average impact are good architecture metrics to monitor for rigidity. System stability measures the percentage of elements (on the average) that would not be affected by a change to an element. Average impact for an element is calculated as the total number of elements that could be affected if a change is made to this element (or the transitive closure of all elements that could be affected).
  2. Fragility: when a change is made to the system, bugs appear in places that have no relationship to the part that was changed. This leads to modules that get worse the more you try to fix them. In this case, these modules need to be redesigned or refactored. Cyclicality metrics can help find fragile modules. Cyclicality is useful in determining how many elements of a system are in cycles. See our blog post “Cyclicality and Bugs” for more information.
  3. Immobility: this is when a component cannot be easily extracted from a system, making it unable to be reused in other systems. If a module is found that would be useful in other systems, it cannot be used because the effort and risk are too great. This is becoming a significant problem as companies move to microservices and cloud-ready applications. A metric that is useful in this case is called coupling. Coupling is the degree of interdependence between software modules; a measure of how closely connected two routines or modules are and the strength of the relationship between modules.
  4. Viscosity: this is when the architecture of the software is hard to preserve. Doing the right thing is harder than doing the wrong thing (breaking the architecture). The software architecture should be created so it is easy to preserve the design.
  5. Needless complexity: the architecture contains infrastructure that adds no direct benefit. It is tempting to try to prepare for any contingency, but preparing for too many contingencies makes the software more complex and harder to understand. Architectures shouldn’t contain elements that aren’t currently useful. Cyclomatic complexity metrics can help diagnose this problem.
  6. Needless repetition: this is when an architecture contains code structures that are repeated, usually by cut and paste, that instead should be unified under a single abstraction. When there is redundant code in software, the job of changing the software becomes complex. If a defect is found in code that has been repeated, the fix has to be implemented in every repetition. However, each repetition might be slightly different.
  7. Opacity: this is when the source code is hard to read and understand. If source code is the design, this is source code that does not express its intent very well. In this case, a concerted effort to refactor code must be made so that future readers can understand it. Code reviews can help in this situation.

Summary

While source code may be the design, trying to figure out the architecture from the source code can be a daunting experience. Using architectural analysis tools like Lattix Architect can help by visualizing the dependencies. This allows you to refactor the architecture, prevent future architectural erosion, and provide metrics like system stability, average impact, cyclicality, coupling, and cyclomatic complexity.

1. C++ Journal, “What is Software Design?”
2. Agile Software Development, Principles, Patterns, and Practices, Robert Martin

Architecture Erosion in Agile Development

Software architecture erosion refers to the gap between the planned and actual architecture of a software system as observed in its implementation.1

Architecture erosion is a common and recurring problem faced by agile development teams. Unfortunately, the process of solving this problem is usually ad hoc or very manual, without adequate visibility at the architecture level. One effective solution is the reflexion model technique. The technique is a lightweight way of comparing high-level architecture models with the actual source code implementation while also specifying and checking architectural constraints.

The diagram below is an example of the reflexion model technique.

Agile Architectural Analysis

Architecture erosion can result in lower quality, increased complexity, and harder-to-maintain software. As these changes happen, it becomes more and more difficult to understand the originally planned software architecture. This is particularly important in an agile environment where, according to the Agile Manifesto, working software is valued over comprehensive documentation and responding to change is valued over following a plan. In reality, this means that the architecture is evolving as the software is evolving. Therefore, software changes need special attention (architectural assessment) from software architects. If this does not happen, the architecture could erode or become overly complex. Uncontrolled growth of a software system can lead to architectural issues that are difficult and expensive to fix.

How to avoid architecture erosion

Architecture erosion can be avoided or corrected by continuously monitoring and improving the software. Continuous checking of the implemented architecture against the intended architecture is a good strategy for detecting software erosion. Once architectural issues have been found, refactoring should be used to fix them. In an agile environment, you should combine development activities with lightweight continuous architectural improvement to avoid or reverse architecture erosion. The process of continuous architectural improvement can be broken down into four steps:

  1. Architecture assessment
    1. Identify architectural smells and design problems
    2. Create a list of identified architectural issues
  2. Prioritization
    1. Decide the order in which the architectural issues will be tackled starting with strategic design issues or high-importance requirements first
  3. Selection
    1. Choose the appropriate refactoring pattern to fix the issue. If none exist create your own.
  4. Test
    1. Make sure the behaviors of the system did not change
    2. Update the architecture assessment to make sure you fixed the design problems and did not introduce new issues. Watch the Lattix Update Feature video for more information on this step.

This is particularly useful in agile development. In a scrum environment, architecture refactoring should be integrated into sprints by adding time for refactoring both code and architecture. During the sprint, architects need to check their architecture, while testers and product owners should validate the system still meets requirements. Architecture refactoring should be done once during a sprint as opposed to code refactoring, which should be done daily. If it is done less often, fixing architectural issues involves more time and complexity as more code changes are added on top of design issues. If done more often, the architecture could change needlessly and add to software complexity. Architectural problems not solved in a current sprint should be saved and maintained in a backlog.

Summary

Architecture erosion can happen in any software project where the architectural assessments are not part of the development process. Architectural refactoring makes sure wrong or inappropriate decisions can be detected and eliminated early. One of the principles of agile development is "maintain simplicity." Focus on simplicity in both the software being developed and in the development process. Whenever possible, actively work to eliminate complexity from the system. A clean architecture eliminates complexity from the software while a lightweight, reflexion technique compliant tool like Lattix Architect makes the process of continuous architecture improvement simple.

1. Terra, R., M.T. Valente, K. Czarnecki, and R.S. Bigonha, "Recommending Refactorings to Reverse Software Architecture Erosion", 16th European Conference on Software Maintenance and Reengineering, 2012

Architectural Flaws: The Enemy Of Software Security

“Microsoft reports that more than 50% of the problems the company uncovered during its ongoing security push are architectural in nature. Cigital data shows a 60/40 split in favor of architectural flaws.”
- Gary McGraw

Nearly 40% of the 1,000 CWEs (common weakness enumeration) are architectural flaws. Architectural design in secure software is an often overlooked aspect of software development. So much so that the IEEE established a Center for Secure Design and released a document “Avoiding the Top 10 Software Security Design Flaws”.

Static analysis is not enough

The static analysis testing of software source code is necessary but not enough. Architectural flaws are difficult to find via static analysis. Architectural flaws can obscure coding bugs that static analysis might have otherwise detected because of the added complexity. Research from Rich Kazman at the Software Engineering Institute shows that you should focus on identifying design weaknesses to alleviate software bug volume. In identifying structures in the design and codebase that have a high likelihood of containing bugs, hidden dependencies, and structural design flaws, SEI has found that architectural flaws and security bugs are highly correlated (.9 correlation). This is because defective files seldom exist alone in large-scale software systems. They are usually architecturally connected, and their architectural structures exhibit significant design flaws that can propagate bugs among many files.

Example HeartBleed

In his essay “How to Prevent the next HeartBleed” David Wheeler said “OpenSSL uses unnecessarily complex structures, which makes it harder for both humans and machines to review.” There should be a continuous effort to simplify the code. Otherwise, just adding capabilities will slowly increase software complexity. The code should be refactored over time to make it simple and clear while new features are being added. The goal should be code that is “obviously right,” as opposed to code that is so complicated that “I can’t see any problems.”

As we stated above, this is a good example of static analysis techniques not being enough. These techniques that were supposed to find HeartBleed-like defects in OpenSSL were thwarted because the code was too complex. Code that is security-sensitive needs to be “as simple as possible.” Many security experts believe using tools, like Lattix Architect, to detect especially complicated structures and then simplifying those structures is likely to produce more secure software. Simplifying code is a mindset. There needs to be a continuous effort to simplify (refactor) the code. If not, architectural erosion starts to happen as you add capabilities and slowly increase software complexity.

As David stated above, the goal should be code that is obviously right, as opposed to code that is so complicated that you can’t see any errors. I think Rus Cox said it best when talking about HeartBleed and complexity: “Try not to write clever code. Try to write well-organized code. Inevitably, you will write clever, poorly-organized code. If someone comes along asking questions about it, use it as a sign that perhaps the code is probably too clever or not well enough organized. Rewrite it to be simpler and easier to understand.”