One of the wonderful things about open source software is that you can scratch an itch. Is Something broke or do you wish it did more? You can fix it! But with the latest adoption of several Extreme Progaming techniques in open source projects it occured to me that XP, and in particular Test Driven Development (TDD), as cannocally formed does not work well for external contributions.
Take my two latest patches, one to Marathon and another for the Parallel task on Ant (apparently I am sill a committer, [note to self get login accout pasword...]). Marathon is hardcore on TDD and at Ant it is a Very Good Thing (but not always a deal breaker). With both of these patches I updated the tests, but should the committers take my word for it that they are valid tests?
One of the basic principals of TDD is you write the test first, then run it, and watch it fail. This is before you add the new functionality. It seems like a sublte point but it is a very essential step: it validates the test can fail. It may seem unnesscicary, but it can be critical. All of the other constraints in the system can conspire to make a test uesless. Without validating a failire it is quite possible that you can accidentally code the equivilent of if (false) { fail("I'm broke"; }
. Is that a valid test case? Maybe if you are testing the compiler and it is supposed to choke when it encounters dead code, but cases like that are never that clear to read.
So how does a project handle the case of patches for TDD? In the case I sent to Marathon it was simple, it was broken functionality and it added no new APIs. In this case you apply the patch that has the test first, assert the test fails, and then you apply the rest. But what about more complex cases? Like when the patch is adding new functionality? My proposed solution is submit two patch files, the first one adds the tests and needed API changes needed to compile, and none of the new code for those APIs. The second patch actually contains the code that makes the APIs work. With this the core commiter can take the two patches, apply the first, clean, compile, test, and look at the new failure and say "this is good." Then (and only then) does the committer apply the second patch, clean, compile, test, and see no test failures (or only failires that were there before the patches were applied).
Generating two diffs from the CVS is a pain, but if you are practicing TDD anyway then you could grab your first patch after the test starts failing. This makes developing a two step process, but TDD is already a two step process: (1) create failing test (2) make test pass. Test Driven Patching reflects that as well.