How to Fail at Software Development

10. Is a sane test plan involved?

Score 0 if someone is dedicated to testing and will take a module from a developer any time and test it. The tester will have a setup for testing and will plug the software into the test environment and run it around the block. This should happen more often than it does, but it is very difficult to find someone who is both knowledgeable enough to do this and is also willing to do it. Oh well.

Score 1 if testing is being done throughout development, but the testing is only done at the top level. Individual modules are rarely tested. This is the most common mode of ongoing testing and is okay, but it will always fail to test some special cases. Some of the more sensitive modules are tested for special situations because the test data is chosen well and will stress critical portions of the code. This is probably the best situation you can hope for.

Score 2 if the testing is all at the higher level, but it takes a long time for the tester to get around to running the test on new modules. This not only lets the code age in the programmer’s mind before a problem is found, but newer pieces of code may have been based on some other piece with a fatal flaw. This is getting to be more like the real world than either of the two previous cases.

Score 3 if all the testing is at the high level and is done in such a way that some things are simply not tested, they are going to be missed. The real difference between this case and the previous one depends on the capabilities of the tester.

Score 4 if most of the testing is being delayed until the end of the project. With this approach, the tester is doing some testing as things go along, but the main job of the tester is to develop a complete test procedure to be followed at the very end of the project when the software is complete. The ongoing portion of the testing is minimal and actually takes place only as parts of the system are completed and put into the final release. The development of the final testing procedure is done in parallel with the software development with the intention that the two be completed at the same time. This means that the test procedure and the software must both be modified in phase with every change to the software specification. The reality is that the test procedure will require almost as much debugging as the software itself.

Score 5 if there is no testing done. Or none is planned to be done. This means that every programmer is responsible for testing his own stuff, and that can never work. A programmer will only test for things that he knows he wrote code to handle, and the test will be performed in the way his code handles it. Even if system testing is planned, and all system testing is deferred to the end of the project, it will never happen—at the end the project will be so far behind schedule that the testing will be skimped or skipped.

Chapter 15 of How to Fail at Software Development describes how you can design and implement tests in such a way that they will have no effect whatsoever, except to consume time.