Mutation Testing
Mutation testing evaluates test suite quality by introducing small code changes (mutations) and checking whether existing tests detect them.
In Depth
Mutation testing works by systematically modifying the source code (e.g., changing + to -, swapping >= to >, removing a require statement) and running the test suite against each mutation. If tests fail on a mutation, it is 'killed' - meaning the tests caught the change. If tests pass despite the mutation, the test suite has a gap. This technique is invaluable for evaluating invariant test suite completeness.
Frequently Asked Questions
What is mutation testing?
Mutation testing evaluates how good your tests are by introducing small bugs (mutations) into the code and checking if tests catch them. If a mutation survives without any test failing, your test suite has a gap.
How does mutation testing relate to invariant testing?
Mutation testing tells you if your invariants are strong enough. If you change a + to - in the contract code and no invariant breaks, your invariant suite is missing a property that should catch that type of bug.