Lesson 7.6 Testing Your Programs
Why test your code and how should you test your code? Beginning programmers often think that testing is the process of proving that your program works correctly and contains no errors. Nothing could be further from the truth.
As the famous computer scientist Edgar Dijkstra wrote:
"Testing can prove only the presence of errors, never their absence."
The purpose of testing is not to prove your code error free, but to bring the errors in your code to the surface.
In this section, you'll learn:
- What kind of tests you should perform.
- How to decide when to stop testing.
What to Test
Testing means systematically running your program using different inputs, and then comparing those inputs against output that is known to be correct.
For instance, the Unit Wrap-up in Lesson 5, "Debugging", required you to fix the program DaysInYear. How did you test the program to see if it worked correctly?
Exhaustive Testing
One option is to just check every value. If you had an almanac, you could start with 1900 and compare every year against the number of days for that year recorded in the almanac.
I don't see any hands raised, so I'll just assume that none of you attempted this form of exhaustive testing. Exhaustive testing sounds simple but is completely impractical for all but the smallest data sets. Simply testing every possible combination for a 10-byte input field would take almost 4 trillion years, running and checking ten million cases per second.
Representative Tests
Most testing relies on identifiying a number of representative tests, instead of exhaustive testing. A representative test is a test that "stands in" for the larger group.
Here's an example. Suppose you have a method called isOdd(), that tells you whether a number is odd or not. To test your method you'd certainly want to see if it works by sending it a few odd numbers, and making sure that it returns true.
You shouldn't stop there, however. What if the method returned true for every number? In addition to the odd numbers, you'd want to pass some even numbers as well, to make sure your method could correctly identify them.
After testing and verifying that your method can correctly separate even from odd, you may think that you are done. To be safe, however, you should run at least three more tests:
- Check that zero and negative numbers are handled correctly.
- Check that very large and very small numbers are handled correctly.
- Check that incorrect values are handled appropriately if it is possible for incorrect values to be entered.
Geometric Relationships
Of course, the isOdd() method is relatively easy to test because there are only two possible outputs--true or false--and the relationship between the inputs and outputs is well established and clearly understood.
With the addition of more inputs, a wider range of outputs, or a more complex relationships between the various inputs and outputs, things can quickly become more complicated.
One technique that is frequently used is to first identify a valid input range, and then create a test suite using values that have a "geometric" relationship to each other. You certainly want to test:
- values at the beginning and end of the range.
- values immediately below and above of the range.
- values clearly inside the range.
- zero and negative numbers, if input is numeric.
- the empty String if input is a String.
- the value null if input is an object or String.
Here's an example of a set of representative tests you might use for a menu system that only accepts the numbers 2-5 as input. [Of course, with only four input values, you'd test them all. However the principle used to decide which values to test would still apply if there were 400 or 4,000 values.
When to Stop
For most software, testing stops when the scheduled ship-date arrives. However, how can you tell when you've thoroughly tested your program? Simple. You measure the amount of coverage.
What, you wonder, is coverage? Coverage is simply a way of measuring how much of your program has been tested. There are two ways to measure coverage. These are called statement coverage and condition coverage.
Statement and condition coverage belong to a different class of testing techniques--a class called white-box testing. When using representative tests, you run your program with different inputs, and compare its output to the expected values.
You don't look inside the program; your program is treated as a black box. Coverage testing is different. In coverage testing, you "open up" the black box and directly examine the code.
Statement Coverage
If your program contains statements that have never been executed, then, by definition, those statements have not been tested. Statement coverage tries to measure the percentage of your program statements that have been executed during testing.
To achieve 100% coverage of all statements--or as close to 100% as is feasible--you start by examining the logic of your program, and then try to come up with cases that will cause each statement to be executed. Note that this is different from simply choosing representative cases. You should add the tests discovered by statement coverage analysis to those you are already running as part of your representative tests.
Condition Coverage
Condition coverage measures the number of branches in your program that have been tested. Remember, an if-else-if statement is a single statement. Just because you've tested every statement in your program [100% statement coverage] doesn't mean that you've tested every line in the program.
A program that has achieved 100% condition coverage is one in which the condition part of every if, switch, and loop has been both true and well as false. That means there are no conditions in your program where only the true case or the false case has been tested. An added bonus of condition coverage is that, if you fully exercise each condition, you will automatically achieve 100% statement coverage.
Something to Talk About
Look at DaysInYear.java, the applet at the end of the last section of this lesson, The Debugging Process. What input values are required for:
- a set of representative tests.
- tests necessary to achieve 100% statement coverage.
- tests necessary to achieve 100% condition coverage.
This is the last section of this lesson.
|
