Filed under: Technical Debt

Using Sonar to monitor and remediate code bases

Steering a Super Tanker is an excellent presentation that shows some innovative way to identify and attack technical debt. At the heart of this presentation is the notion of identifying and remediating modules with high complexity per method, an approach I too would advocate. However I find it curios that the motion chart axis are set to complexity/method in the Y and LOC in the X with color and size set to complexity/method and overall complexity! The beauty of the motion chart is it's 4 axis representation so I find the size and color setting to be redundant. It's no coincidence that the color exactly correlates to the Y axis while there is also a strong correlation between overall complexity and LOC. It's not until the Gaming the Metrics section that the author comes close to a setup I use everyday. I find that setting the Y axis to Coverage, X to rules compliance with Color set to Complexity per method and size to LOC provides an excellent overall view of the modules. With this arrangement we can see zero-test modules and identify any high complexity items as risk. So long as our highly complex items also have high coverage then we might not have to panic so much.

The other really great thing about this view and the Motion Chart in general is the ability to spot things over time. At my last job where I was monitoring millions of lines of code accross 50+ projects I would periodicaly view the motion chart, in motion. This video playback representation of the data over the previous three months frequently allowed me to spot interesting events within projects. One great example that I love to cite was when I witnessed a sharp increase in coverage on a couple of the hundreds of modules under inspection. First one, then a second and third module shot up the chart (coverage) and then drifted to the right (rule compliance). All three modules where a part of the same project and prompted me to reach out to the project owners to commend their efforts and learn how it come to be. What I learned was the team had installed a lava lamp to indicate broken builds and the business owners had come to be aware of the lamp upon seeing it lit up. The short story is this lamp then led to a discussion as to why the build was broken and ideas about how to improve. The developers had felt like they weren't funded to write to unit test but explained how they could have helped identify issues in local builds before the code was committed. So, the business added a Unit Test coverage target, along with a rules compliance target, to thei non-functional requirements thus authorizing the developers to spend time on these task. Within 2 weeks the three modules had 80%+ coverage and improved their rules compliance by 15-20%!

While the Motion Chart is a great visualization I'd also suggest keeping a close eye on Clouds->Top Risk and Hotspots. Clouds offers a much more granular view and can spot complex/low coverage packages that can be hidden in high LOC components. Hotspots provides a no-nonsense, actionable view of potential areas to dive in one.

In general, the biggest issue I battle with Sonar implimentations is that people really want to be *told* what to fix. It takes time to get folks used to Sonar reporting and to establish a culture that knows how to drill in for themselves based on the issues at hand. Sonar provides so much data it can be overwhelming at first. I've found that focusing on Coverage in the Most Complex or Most Used modules/packages is a good place to start for legacy apps.

Greenfield apps should establish non-functional requirements for Rules Compliance and Coverage and consider using the SQALE model to focus efforts by life-cycle. Use sonar after each sprint/iteration and/or release to continualy monitor, talk about and remediate issues as they appear.

Technical Debt as a Business Goal

Media_httpwwwenerjyco_fubul

As Figure 1 demonstrates, a Java file with Cyclomatic complexity of 38 has a probability of 50% to be error-prone. If you do not stop it prior to deployment through technical debt criteria, it is likely to affect your customers and play havoc with your deployment quite a few times in the future. The fact that it did not do so during the first hour of deployment does not guarantee that such a file will be “well-behaved” in the future.

I am going to go back through my *work* related post from Lasterday and bring them over here, revisiting them as I do so.

I remember this chart in particular really grabbing me, the idea that a 50% probability of a defect could be predicited from nothing more than complexity. Could we start to produce *potential defect rates* a la Casper Jones and function points? Even if it has decent testing at say the Unit and Functional levels? Is every 80 points a defect or more?

The idea of tying TD criteria to release criteria is an interesting one though I'd prefer to have *business goals*. If the *business* is the driver then there is no question but if the dev team set's these *technical* thresholds, then how do you rationalize a delay back to the business??

 

What would be some reasonable goals? Minimum 50% unit test coverage?  No more than $2 TD/LOC? Does something like SQALE make the most sense as a way to define and track quality goals?