I’m not sure if other people have had the same issue, but I see more and more developers “fixing the unit tests”, rather than understanding why they are broken and the business requirement they were originally testing.

The problem use to be no-one wanted to write unit tests. 18 months on, and now most developers in our department are writing unit tests, however we have this new problem now.

So, a release has gone into production, and all of a sudden we have a broken build. Let’s assume for this post, that sadly, this project (named: Project Oh Dear) didn’t follow all the processes we have in place, such as unit testing and continuous integration using our Jenkins build platform. It happens!

So, what needs to be done? We need to “fix the unit tests”. However, this statement can be taken literally, where the sole aim now is to make the unit tests pass.. Not really understand why they are broken per se, but get them green again so all is well.

To take an arbitrary situation, let’s say that we had an object that kept an internal array of errors that has occurred during an execution of a script and it was fully unit tested to make sure, on error, the internal array is updated to mark such an event. The requirement here being that the errors are ultimately persisted to some datastore somewhere for later interrogation. OK, so the new project, Project Oh Dear, comes along, and the developers aren’t unit testing for one reason or another, and we now have a requirement to on error, log the information to the terminal. Let’s assume there is a misunderstanding and the developers now believe that the original requirement is defunkt, and we now only care about logging to the terminal. I know, I know, maybe someone should have checked, but you know, everyone’s busy!

Project Oh Dear kicks off and this requirement is started.. We no longer need the internal array to persist the errors, so that’s removed, and the on error event now logs to the terminal.. This feature, along with all the other features enters normal human QA processes and passes, and then rolls live.

Broken Unit Tests!

Fix the Unit Tests the magagement say!

OK, so we update the original unit tests to make sure they capture output, and that the output is what we expect.

Committed to version control and that’s it.

Later down the line however, the orignal developers look over the code and realise that at several stages, Project Oh Dear has subtly changed behaviour, and simply removed/changed valid unit tests.

The orignal requirement of persisting errors to the datastore was valid, the unit tests failed (quite rightly), but instead of understanding why they failed, the developers “fixed the unit tests” to capture the behaviour that is now in production, therefore ignoring the fact that the broken tests were validly broken!

This is a very simplistic arbitrary example of what is occurring, but the principle is the same. Fixing the unit tests, is very much more than simply make them go green. If you are in the situation of “fixing unit tests”, then you probably have a fundamental issue somewhere else in your processes. If you are the developer assigned to fix broken unit tests, please make sure you fully understand what they were testing, what the business requirements were, and why they are now breaking. I imagine that it is very rare that the unit tests are broken, more so that the production code is somewhat broken.