Test it, for documentation sake!

One of the challenges of software development is to have up-to-date documentation of the system design. Knowing how your software works and having this knowledge in a concise and presentable form is crucial. It enables you to communicate on your current situation, about your planned changes, about your problems … and it allows you to pass this knowledge on to new team members. Unfortunately, system documentation is one of the most neglected topics in the enterprise software world.

Why this?

The general view of developers and architects is that there is no value in documentation, since it never reflects the current state of the software. Keeping the documentation current is way too expensive, perhaps even impossible. Therefore, the common reasoning is that working on documentation is a waste of time and energy.

Based on this real life experience, the agile community derived some key points when it comes to documentation. The most outstanding are:

  • document late
  • document only stable things
  • document lean and sufficient, just barely good enough

Those points have a common goal: to reduce the costs by minimizing documentation or avoiding it when things are in flux. But is this really the right way to tackle the problem?

The point is about communication

The whole point in documentation is communication. So what are the odds that you need to communicate about something that probably won’t change anymore? Actually, what are the chances that some part of the system won’t change anymore anyhow? My guess is that the answer to both questions is: not very high.

Instead of focusing on documentation that is irrelevant, we should be focusing on creating documentation that supports us in our daily communication. It should be helping us during the development process and the associated costs should be minimal.

The basic idea

I have been tinkering around with the following idea:
I always found that unit tests are a great way to understand how a class should be used and how it works. Why not extend this concept and create high-level tests specifically for documenting the system design? Since those tests are part of the code they share all the benefits of the regular development process: they can be changed and refactored along with the system so they always reflect the current state. But the best part of this is: since they are runnable tests, they can be used to record runtime information and display it in a presentable form like a sequence or class diagram.

Based on this idea, I created a simple prototype. Let me show you how it works:

Suppose we are building a new online auction system. The system has auction engine class that enable users to place their bids. Lets assume that we have a JUnit test in place for the auction engine:

public class AuctionEngineTest {
public void placeSucessfulBidTest() {
AuctionEngine auctionEngine = new AuctionEngine();
BidData bid = new BidData(auctionId, userId, maxBid);
BidResult bidResult = auctionEngine.placeBid(bid);
assertEquals(BidResult.OK, bidResult);

Now this is a nice test that runs a very important part of our application. It probably is the typical example for the code that you want to have a good overview about what is going on.

Binding documentation to the test

We have our integration test in place. What we do now is to add an annotation that enables the appropriate tracking of the test along with some JavaDoc:

* This test shows the basic flow when placing a successful bid. Notice that before the transaction is started a quick sanity
* check is perfomed.
* {@sequence}
public void placeSucessfulBidTest() ...

When we run this test the @Track annotation is evaluated and a “flight recording” is started. When the test is done the recorded data is dumped to a file. Afterwards, a custom doclet is used to create the HTML JavaDoc along with the sequence diagram of the test. The result looks like this:

And there you go: a documentation of your system design that stays current with your code as long as you keep your tests running.

One Comment

  1. Dragan-Sassler
    Posted June 20, 2012 at 11:10 | Permalink

    A brilliant idea!
    Documentation is a reflection of the test or documentation is just as good, how good is the testing