Discussions: How to deal with Technical Debt in Open-Source Software?

Technical Debt (TD) is one of the most important aspects in today’s software engineering. It needs to be monitored and properly taken care of before it spreads too much within the code base. TD in a software project is more than a need for proper refactoring. It resembles the cost that the company, organization, or team behind a project, is paying to maintain the TD, in the long term. So, if you feel that refactoring is a commodity, a nice to have process for when we “can afford” to spend time on an extra activity, we invite you to read this article. But even if you are respecting TD and you are mindful of the damage it makes to your company and your software product, this article will be helpful for you as well.

For this article, I had the pleasure of interviewing an old friend, Professor Apostolos Ampatzoglou, an academic in the Software Engineering field, with extensive experience on TD, among other research topics. Prof. Ampatzoglou is currently an Assistant Professor of Software Engineering in the University of Macedonia, Thessaloniki, Greece, and former Assistant Professor in the University of Groningen in the Netherlands. In the following pages he will guide us through what TD is all about, how it is linked to corporate costs, what’s the differences of TD between Open-Source Software (OSS) and proprietary software and what can software engineers do to “tame it”.

Q: Prof. Ampatzoglou, it is a pleasure to have you with us. Can you please introduce yourself and explain your areas of expertise?

A: Pleasure to be here, thank you for the kind invitation. My name is Apostolos Ampatzoglou, and I am an Assistant Professor of Software Engineering in the University of Macedonia, in Thessaloniki, Greece. Before that, I was an Assistant Professor in the University of Groningen in the Netherlands.

My main research area is software maintenance and quality assurance, from a quantitative perspective, that is using metrics. In the last decade I started working on TD, for which now I could say that we have a good understanding by collaborating with various industries, but also by studying its accumulation and repayment in OSS systems.

Q: Could you define the concept of "technical debt" (TD) for our audience and explain how it applies to OSS?

A: Technical Debt (TD) is a metaphor from economics that has been transferred to software engineering so that we have a cost-oriented perception of how poor-quality costs (see manufacturing [1]) can be experienced, while developing software. In particular, the typical scenario in economics is that a company that lacks liquidity, asks for a loan in some kind of bank. This solves their short-term problem of lack of liquidity, but as a side effect it creates a long-term financial obligation, termed as interest.

In software development, the company decides to (or even unintentionally) bring the product earlier in the market (either by not delivering it in the perfect quality in terms of maintainability or by taking shortcuts - e.g., duplicating code, long methods, God classes, not appropriate frameworks, etc. -). The amount that the company “saves” from these shortcuts (let’s say, for example, that the company saves 3 man months (MMs)) can be considered as the capital of the technical debt (3MMs x €5,000 = €15,000 of debt). However, due to the poor maintainability, every time that a component that contains debt needs to be updated in the future, the company needs to pay extra effort, compared to the one that they would need to devote if the component was perfectly maintainable. So, suppose that the system changes every month, for 5 years, and in every change the company loses 5 extra hours of work, due to TD. This will account to the following interest: 60 months x 5 extra hours x 50 euros per hour = €15,000 euros. In that sense, the breaking point for this system (breaking point = when you have paid, what you have saved) is 5 years. If the system is maintained for a longer period, then the company has a loss.

A very relevant question that we get when transferring this concept to OSS, is whether this is applicable; in the sense that in OSS, in general, there are no payments or payrolls. In fact, we believe that the whole concept is perfectly applicable, since OSS systems are large enough and are maintained for long periods and therefore can be considered liable to TD. A high-level difference is that the unit of measurement is more fitting to be effort, rather than currency, but this is a small problem compared to the benefits that can be brought by quality control.

Q: What initially drew you to studying technical debt in the open-source world?

A: To be honest, in the very early years of my research, back in late 2000s’, it was a convenience choice, since OSS has offered a large variety of data that we could use and freely experiment with. Nevertheless, even from the beginning of these endeavours, it was made clear to me that OSS was not a playground for developers to experiment and produce small projects that would be of interest only to them (this is a very common mistake that many people make, when they have not dived into the magic of OSS development). As years passed, and I became more familiar with the domain, it was and still is clear to me that, OSS development is equally important to studying TD in industry.

The importance of OSS development and its implications in practice can be seen in two ways:

The fact that OSS can be an accelerator for innovation — you can check a presentation by P. Krief from the Eclipse Foundation [2].
Almost every IT industry that relies on OSS, is brining into commercial software the quality, TD, and vulnerabilities of OSS components, making them both complementary and not competitive factors — you can check the “Software Bill of Materials” from the Software Transparency Foundation [3].

Q: How would you compare the technical debt issues in OSS projects to those in proprietary software?

A: Current empirical evidence suggests that old stories implying that quality management in OSS development is at a lower level compared to industry, are not valid anymore. With respect to quality management, over the years we have observed that some OSS foundations are producing code of excellent quality. In a recent work of the group [4] we have seen that in many projects of the Apache Foundation, the Technical Debt Density (that is the average TD per line of code) is reducing along evolution, resisting to the usual laws of software evolution, stating that quality diminishes, and complexity increases as software grows older and larger. Evidence from other groups [5], that compare TD (along with correctness and security) between proprietary and OSS projects, suggest that in general the levels of identified issues is comparable between the two, given their respective size. Nevertheless, it seems that in proprietary software these issues tend to survive less.

Q: Can you provide some examples of open-source projects that have managed TD effectively? What strategies did they use?

A: Although specific projects do not come to mind, there are quite a few. As mentioned before, there is evidence in the literature that several projects of Apache have reduced Technical Debt Density over time. The main reasons for this can be the existence of specific quality assurance processes. For example: quality control at commit level, commit guidelines, specific discussions on quality during meetings, and in general a culture of quality in the project, etc.

By contrasting low-level factors, such as the characteristics of artifacts and high-level factors, like process-level management, we have seen that macro-TD management is more efficient, since it affects the overall project quality [6].

Q: Conversely, can you share any examples of OSS projects that struggled due to TD?

A: Similarly, I cannot recall projects that struggled a lot with TD. But in accordance with the previous statements, projects that degrade in terms of quality and TD, are projects that do not refactor systematically, that do not impose quality control for new code, and in general that do not have a TD culture. The same applies both in industry and OSS development, we do not see difference into this. As a general guideline, to reduce our TD and make our codebase viable, we either need to prevent TD (maybe through a quality gate) or repay TD (through refactoring).

Q: Are there any open-source software categories that tend to accumulate more technical debt than others? If so, could you explain why?

A: To be honest, although this study would be low-hanging fruit for research, I am not sure if I have seen evidence on such a comparison that focuses on TD. I can only provide some hints and a gut feeling, based on other qualities that we have studied in the past, like the extent of using design patterns in OSS of different categories or the reusability of components in different domains. So, for those quality attributes, there is evidence that entertainment-related software, such as games, audio and video processing software, tend to be of lower structural quality, compared to software development tools, libraries or frameworks, for that matter.

These data, however, are quite old (considering the evolution curve of software). The scenery now maybe be very different.

Q: What are the major causes of technical debt in OSS? Are they different from those in proprietary software?

A: For answering this question, two sources of evidence can be referenced. First, for industrial software there is a research project, called InsighTD [7], which studies the root causes of TD, by surveying developers from multiple countries: Brazil, Argentina, USA, Chile, Serbia, Colombia, Costa Rica, Netherlands, Finland, India, Italy, Norway, and Saudi Arabia. After replicating the same survey across these countries, the team has recorded 78 causes and 66 effects of TD [8]. For data retrieved from OSS, we can refer to quantitative results [4] that, as mentioned before, study if the cause of TD reside at the micro-management or the macro-management level.

Q: How can contributors and maintainers to OSS projects identify and measure TD?

A: Measurement is one of the most important open problems in the TD community. Currently, there are more than 20 tools that can be used for measuring TD [9], but it seems that their level of agreement is very low [10]. This suggests that the market is yet not mature enough to select a state-of-practice tool to be used. However, in terms of existing solutions, at this point SonarQube is the key-player of the game [11], since practitioners and researchers use it the most, compared to other solutions.

Given that the main criticism on TD measurement solutions is not on TD identification (i.e., that a problem exists), but on the accuracy of the actual measurement (i.e., these problems costs X dollars to be fixed), I would suggest that it is safer to use tools for identifying TD items and spot problems that can be fixed in refactoring sessions.

Q: How does the culture and community around an OSS project impact its accumulation and management of TD?

A: These two points are very important. Culture is the cornerstone of TD management. The first step to have a system with manageable TD is to make the developers and managers understand that accumulating TD is a problem. If this is perceived, then building a culture of refactoring through refactoring sessions, or just refactoring before changing the code, is something that can be more easily be “sold” to developers. Similarly, having perceived the benefits of TD, it is easier to develop a clean code mindset, which eventually reduces TD, in the sense that new code that comes into the system will have lower TD compared to the existing one that it substitutes. Or, at least, the components that are changing more frequently (usually the ones implemented more recently) will not generate a lot of interest, since they will be clean. Cultivating such a culture in the community around a project is a challenging task for project leaders, owners, or moderators; rising the need for quality control mechanisms, processes, etc. that need to be defined and discussed in the board meetings. However, we have seen this happening into practice sometimes, with a striking example of a company that hired a quality manager for TD monitoring after a close collaboration in the context of an EU project.

Q: How does technical debt impact the security of OSS projects?

A: This is something that has started to be investigated lately [12], [13]. This is a bit beyond my field of expertise, but I have the feeling that in general, a low TD code is usually less vulnerable. This is because code is more easily maintainable and security mechanisms can be easier to introduce. But as mentioned, I am no expert to this specific research field.

Q: Could you elaborate on any current research or innovations in the field of managing TD in OSS?

A: TD research and innovation is flourishing. In the last years, we have seen and used TD in various research and innovation actions that have received funding from EU: Some of them include but are not limited to: TD for low energy devices, TD for scientific development, TD for service-based systems, TD for HPC. There are more.

In general, I believe that as TD is becoming more and more established, more quality assurance innovations will be built around TD and TD-related audits. In terms of OSS, as mentioned earlier, TD is studied a lot through OSS projects, and therefore a large body of knowledge is being produced.

Q: What do you envision for the future of technical debt management in OSS software? What trends or changes should we expect to see?

A: What I see as two trends, not only in TD for OSS, but in general, are:

How can AI be used to generate TD-free code, or how it can help to reduce TD in existing systems?
Human aspects in TD management.

The rise of ChatGPT and Co-Pilot is a change in the way of developing software that we cannot neglect as scientists and practitioners. Before putting more and more such systems in our routines and processed, we need to be sure of the quality of the code that they produce. In terms of human aspects, the notions of team composition, work modes, code ownership, etc. are important viewpoints that need to be checked in more detail.

References:

[1] https://www.amazon.com/Poor-Quality-Cost-Implementing-Understanding-Reliability/dp/0824777433

[2] https://www.slideshare.net/philippekrief/open-source-is-eating-the-world

[3] https://st.foundation/projects

[4] https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9234106

[5] https://onlinelibrary.wiley.com/doi/pdf/10.1002/smr.2425

[6] https://ieeexplore.ieee.org/abstract/document/10024756

[7] http://www.td-survey.com/

[8] https://link.springer.com/article/10.1007/s10664-020-09832-9

[9] https://ieeexplore.ieee.org/abstract/document/9200792

[10] https://link.springer.com/article/10.1007/s10664-020-09869-w

[11] https://www.sonarsource.com/products/sonarqube/