Tests are often treated as write-once-never-look-at-again code (unless just maybe if there is a bug). That is a shame, because it wastes much of the potential value tests can provide. Good tests are helpful for understanding a piece of code, and making code easier to understand is important since code is harder to read than to write.

To help make code easier to understand, I suggest that you should start looking at your tests as a living documentation of your code. View them as something that your colleagues will look at to understand the code. See them as something that you will look at to understand the code (later).

Focus on communication and intent

Why? Well, obviously, because they hopefully help you understand the code quicker. But it goes deeper than that – I believe it helps you write better implementation code. The reasons is that it helps you focus on communication and intent, rather than technical details. Because after all, the most important audience of code is not computers, but humans. Computers understand any code that is correct, no matter how it is structured. The same cannot be said for humans. You turn computer-focused tests into human-focused documentation.

Behavior-Driven Development go with this theme to turn tests into a collaboration tool between software developers and business experts.

Seeing tests as stories is an idea I learned from Kent Beck, JUnit’s creator, who said:

Writing a test is really comes down to telling a story about the code. Having that mindset helps you work out many other problems during testing.

What documentation-focused tests look like

A few examples of how viewing tests as documentation can affect your test writing.

• Your start using longer, more descriptive method names.
• You name test methods not only after the method they are testing, but describes what to expect.
• You use multiple test classes for the same implementation class, separating groups of related tests. (These groups are called “fixtures” and are how JUnit was designed and intended to be used.)
• You start sorting test methods in a way that makes them easy to grasp.
• You accept somewhat higher duplication in your tests because readability of tests is so important.
• You avoid abstract super classes for test classes as they make individual test classes harder to read.
• You write tests before implementation code to help you focus on the purpose of the code.
• You realize that the tests needed to drive design in Test-Driven Development are not necessarily the tests needed to show the intent of the code.
• You understand that deleting tests comes down to story telling – which tests are needed to tell the story about this code.
• You don’t obsess over 100% test coverage since you get less in return for each test you add.

Caveat: Of course, there is also a technical or coverage aspect of unit testing, but in most cases it is of less importance. The primary audience of code is humans, not computers.

StoryTeller to the rescue

If you want some help with the mind shift, that is where StoryTeller comes in. It is an Eclipse plugin that uses your unit tests as living documentation.

It helps you by displaying relevant tests side-by-side with your code, and displaying test names as normal sentences rather than method names.

By always having relevant tests available, you can use the tests as a quick reference of the code. You also get reminded about the tests regularly, so you can see if they accurately describe the code. If not, it might be time to add, modify, or remove a few tests.

StoryTeller also uses normal sentences rather than method names. WhileCamelCaseIsVeryUseful, plain old text is just easier to read. Using normal sentences also helps you think of the tests as specification or documentation, not just methods.

If you use Eclipse, go ahead and give StoryTeller a spin. You just might like it!

We write tests to help us rather than making our lives more difficult. However, unit testing and Test-Driven Development have often been advocated religiously – “you must do it 100% or you’re doing it wrong”! But that is not true.

If you look at the unit tests you’ve written the last year, how many of them have actually helped you? How many of them has caught a regression? How many helped a new programmer understand the code its testing? How many would still work if you refactored the code under test? I could go on. Unfortunately, in all likeliness, many of the tests are a waste of time and should be removed!

Probably the greatest misunderstanding regarding Test-Driven Development is that people focus on the “Test” part, when the latter words “Driven Development” are much more important. TDD is meant to help us drive the development of our system forward through tests, not to produce a fool proof set of tests. Of course, there are cases where we want to ensure that some complex code works as expected, but that is simply testing, not test-driven development. If you want to verify that some code works, by all means, write unit tests for it. If you want to use Test-Driven Development to help you develop your systems, write the tests that help you do that.

Especially, I think there are many cases where one test too many is unnecessary, or even harmful. Here are a few examples of such situations.

• The cost if the code breaks is very low. In some situations, a bug doesn’t cause very much problems. Most people doesn’t write unit tests for their shell scripts, for example.
• There is something more important to do. If a customer can’t even purchase your product, it doesn’t matter how good the unit tests for canceling an order are.
• The code is too simple to reasonably fail. In many cases, the code is very unlikely to ever fail. The extremely occasional failure will most likely cost much less than writing and maintaining unit tests for it.
• Setting up the test requires too much work. If you depend on something complex which is hard to fake, don’t waste your time writing that low level test. You’ll most likely fake the dependency incorrectly anyway. Leave it to a higher-level test.
• The unit under test is a small private helper class. When the class you’re looking at is just a small private helper class for some bigger public class which performs some real business functionality, you probably don’t need to test the helper separately.

In the examples above, the tests does not help us drive the development of our system forward. Instead, they slow us down. So focus your energy on writing tests that help you  drive your development forward. Write tests for new features, for learning, for things you expect to break, for important edge cases, and for bugs that you fix. Beyond that, don’t write more tests.

I like the answer from Kent Beck on a “how much to test” question which sums this whole topic up rather nicely.

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

A test is meant to help you. If it does not help you, something is wrong.

There are many ways in which a test can be a hinder rather than a help. It can be tightly coupled with the implementation, making the code harder to work with. It can be long, complex and very hard to read. It can execute some code without actually validating anything, giving a false sense of security.

If you find yourself having a unhelpful test on your hand, I would recommend taking one of the following steps, in this order.

1. Understand it. Make sure that the problem lies with the test, and not your understanding.
2. Fix it. If you do understand it, and realize that it is a badly written test – fix it!
3. Delete it. As a last resort, if fixing the test is not worth the cost, the test should be deleted!

Positive Return On Investment

To put it another way, we can borrow some terminology from economy: a test should have a positive Return On Investment (ROI). That means as follows.

The value you get out of a test should be higher than the cost to write and maintain it.

If this is not the case, the test needs to be fixed or deleted.

Keeping a bad test just because “it’s already written” is a dangerous road to take. Tests still cost money to maintain. If it is in a part of the code that isn’t changed anymore, fine, keep it. If it tests code that is under development, then do something about it.

Recognize the following scenario? You get a description of a non-trivial problem. Before you’ve even heard the full description, your problem solving brain starts working on a solution. After a few quick adjustments of your idea, you feel you’ve successfully solved the problem and start coding.

I bet you recognize it. I bet you do it all the time – I sure do. If you have experienced this, I’m sure you’ve also noticed that this first attempt at a solution tends to be incorrect, especially for more complex problems. Often you realize later that you didn’t completely understand the complexity of the situation.

This is one of the reasons why I choose to write unit tests before implementation –  to ensure that I understand the problem before I decide on a solution.

Test-Driven Development is about design, but also learning

As you perhaps have heard, Test-Driven Development (TDD) is not really about testing, it’s about design. That is true. It encourages you to think about your  code from a usage perspective first, rather than an implementation perspective. You write code that uses the implementation-to-be before you write the actual implementation. The interface of the code to be written is colored by how you want to use the code, rather than how you plan to implement it.

While Test-Driven Development is about design, is also about learning. It makes you to think of the problem before the solution. I find the following rule of thumb helpful.

If you cannot describe the wanted behavior in a test method name, you are not ready to start coding.

If you follow this rule, you vastly reduce the risk that you waste your time on implementing a premature solution.

A map reflects the ones understanding of the terrain.

Implementing the first solution that comes into your head would be like running as fast as you can while looking at your feet, not realizing if are running the wrong way. When you later realize that you’re not were you wanted, you will have to choose if you want to live with being in the wrong place, or spend even more time going to the right place. It’s more effective to pause for a few minutes, and then run to the right place from the beginning.

Another way to see it is that your understanding of the problem is your map to the problem. The more correct your map is, the more likely you are to end up where you want to.

Despite our best efforts, every now and then a bug slips through. Sometimes it is caught early in our own testing, sometimes it goes all the way into production.

What happens when the bug is found? Hopefully, it is fixed, the fix is verified, and a new version is released. This happens all the time. In some organizations it is a formal process with multiple gates, in some it is a very informal one. The part which I find interesting is, if for some reason this bug was to appear again, how long time would it take to find and diagnose it?

A bug should only be found by a human once

If a human is involved in catching a regression, I believe you have failed! In my mind, finding the reappearance of a known bug should be a completely automated process. That means, for every bug you fix there should be an automated test which will fail if the bug reappears. It doesn’t matter if that test is a unit test, an integration test, a system test, or any other kind of test. The important part is that it is automated.

I don’t know how many times I’ve had the feeling of “didn’t we have this problem a few months ago?” only to later find out “yes, we did have this problem a few months ago.”

So, my suggestion is rather simple.

For each bug you fix, write an automated test to prove it!

If you do not do this, you either waste a lot of time by doing manual regression testing, or you are unprofessional enough to let your users be your testers.

A nice side effect of this practice is that over time you build a rather robust regression test suite for known, actual bugs. The tests also give you a form of documentation for all the special case handling that is often the mark of a battle-hardened system.

Most of us want to develop with quality. We want to create things that are really good, that work well, and that delight users and customers.

In order to do this, we come up with a number of practices, rules, policies, and other tools to ensure we stay on a good path. While all these can help us, they can also cause us to lose focus on the real goal. “You get what you measure”, is a common saying, one that also means that if you measure X in order to get Y, you will receive X but not Y if you push the measuring hard enough.

What I would like here, is to encourage you to take a step back, think about what you are doing and why. Is the thing that you are doing helping you reach “good” or just “done”? Are you improving the system or just extending it? Another way to put this, I want us to work on realizing a vision rather than a requirement specification. Everyone from developers and testers to architects and project managers should know and care for that vision. They should also have mandate to try their best to realize it.

As an inspiration, this is the “requirement specification” that Michelangelo received for his frescoes in the Sistine Chapel.

Please paint our ceiling for the greater glory of God
and as an inspiration and lesson to his people.

He did also receive a few suggestions for motives, but the above quote was the vision that Michelangelo was asked to realize. I think we can agree on that the result was more than “ok”!

The ceiling of the Sistine Chapel