jeff.dallien.net

I’m not a fan of pending specs or, more generally, tests which are present that aren’t being run. When I come across a pending spec, it raises questions but offers no answers.

• Why isn’t this test being run (not always clear from the reason given)
• Is this broken or missing functionality?
• Is this a broken test for working code?
• Was it to test functionality that has been removed?

Ideally we would all use pending specs sparingly, to remind ourselves of things which the application needs to do that we are going to work on soon. In practice with a team of developers this doesn’t always happen. Sometimes a developer doesn’t want to expend the effort to find why a test is failing. A bad idea, of course, but it happens. If you care about your tests you’ll want to know when it happens. Sometimes a (possibly empty) pending spec was written early by a developer thinking that it will probably be required, only to find out later that it wasn’t. Sometimes things like bugs in our tools or larger problems in our application prevent us from testing something we know should be tested so we mark the spec as pending.

When these pending specs are ignored, the uncertainty increases. People forget why something is pending. Everyone just gets used to see the messages go by and having those pending specs there just becomes normal.

I want as little uncertainty as possible. How do we fix the situation and prevent it from happening again? A good way is to make it obvious to team members when specs are marked pending. If everyone notices right away, perhaps someone knows a solution and it can be sorted out immediately instead of weeks or months later.

Initially, to keep a watch on our pending specs, my team at MDLogix started recording our number of pending specs during iteration retrospectives. By comparing our current number from the previous iteration’s we would know that at least we weren’t increasing the uncertainty. When we started tracking the numbers we had 51 specs marked as pending. This only accounted for about 1.5% of all our tests but every one of them could have been hiding something that was broken.

Our next step was to schedule time to actively review our pending specs to decide what action we should take on each one. Doing this was not an immediate need but we felt it would improve the quality of our code and tests. To make sure it got done we decided to create a regularly scheduled block of time for our “important but non-urgent” tasks. For two hours after lunch each Friday all team members work on the same task, which we identify in our stand-up meeting that morning.

We found that some pending specs were for functionality that had been removed when our code base had been extracted from an older larger application. Some actually hid problems that required fixing in our tests and code. Our plan is to reduce the number of pending specs in all the projects for which the team is responsible. Once this happens it will be apparent to all the team members when specs are made pending and we can all make sure the use is justified. We will make sure that any new pending specs will not be neglected!