Good Tests Gone Very Bad

I've been writing dozens of tests lately as I have completely bought in to the TDD methodology. I've even turned on code coverage in Team Suite to see just how I'm doing. This may not be a great thing to do if you suffer from OCD, but it sure has helped me to better understand my code. It helps even more when I am walking through other people's code trying to debug something, as I run their tests first, check to see if any lines of code haven't executed as expected, and then write a test to try to execute that code as I would expect. If I still don't get there, I know one of the core assumptions elsewhere is wrong. Its a lot faster than stepping through hundreds of lines of code. It focuses my attention to the problem area much more quickly.

Yesterday morning, I was flying along, writing code, and writing tests, and one of the other developers here came by to ask how to properly test his ViewModel that depended on checking various items on the file system (i.e. does a directory exist, if not, create it).

We're using Rhino Mocks, not TypeMock isolator, so the fact that these System.IO classes are static means that we can't mock them. A little searching revealed that our best option was to create an IFileManager interface with a FileManager class under it, and mock the interface. Not my favorite thing to do, but it works.

I wrote the tests for the new class, and they ran fine.

Then we added that class to my co-worker's class and exposed a property on his interface / class for the IFileManager so that we could pass in a mock.

I ran his tests for his class in debug mode. It worked. Cool.

We wrote some new tests to pass in the mock, and ran those in debug mode. They worked as expected. Awesome!

But here's where things got hairy.

We went to my test list editor, and added them to our global test list, and reran the tests. The failed with a 'Could not load type error'. Hmmm. Must be something wrong with my set up / tear down. So I reran just one of the tests from the test list editor. Still failed. Okay, back to the code. Rerun the test in debug mode. It works. Run the test in 'Run' mode. It fails. WTF?

Check my tests on the FileManager class. They work fine.

Strip the new code out of the ViewModel class. Run the original tests. Pass.

Run the new tests. Fail. Same error. Wait... the class it says it can't load is the class we have completely commented out. Except we left in the using statement. Comment that out. Build. Still failed, same message.

At this point I'm getting pretty PO'd. I force a clean on my whole solution, and rebuild. Tests pass. WTF?

Start adding code back in. Add the using statement. Pass.

Add the reference to the interface. Failed.

Pull the mocking code out of the tests, pass in a real object (not generated by the AutoMockingUnityContainer). Pass. Okay, it's somewhere in the Unity Code. We're now 2 hours into this. I went through this a dozen times, various combinations, until it finally hit me.

The problem wasn't that it couldn't load my FileManager class. By this time, that class had nothing left in it beside a couple of boolean functions that always returned true. What had happened was that immediately before I had been asked to look at this issue, I had installed the October 2008 release of the Enterprise Application Library (Version 4.1) to make use of the Validation Application Block. I had previously just installed the Unity Framework separately. I now had two versions of the Unity Framework installed on my machine, with the same version numbers, but from different locations on my machine.

I uninstalled the Enterprise Library, restarted my IDE, and ran my tests successfully. I readded all the code I had commented out, and the tests ran as expected. I think we spent three hours on this one yesterday (x2 since two of us were trying to figure it out)

Now I have to decide when to have the team install the app blocks on thier machines, and to remind them to unistall Unity and any old versions first.