Automatically Generating NUnit Test Cases

I recently came across a need to populate the TestCase attribute of an NUnit test case programatically.
In case you’re not familiar with TestCase, here’s an example:


    [TestCase(1, 1, 2)]
    [TestCase(1, 2, 3)]
    [TestCase(2, 2, 4)]
    public void APlusBEqualsC(int a, int b, int c)
    {
        Assert.AreEqual(c, a + b, $"Expected {a} + {b} = {c}");
    }

As is hopefully clear from the sample above, TestCase lets you vary the inputs to an NUnit test method while reusing the test code itself.
For each TestCase attribute supplied, a new test is created using the supplied parameter list.

But what if you want to generate these TestCase attributes through code?

An unambiguous source

My use case came about whilst writing integration tests to exercise data upgrade scenarios. The tests drop data satisfying an old schema into the system and then exercise certain key scenarios that would require a successful upgrade. I wanted to prove that all previously released schema versions can be upgraded through the current codebase successfully.

I can use TestCase to specify all of the data versions that I want to test upgrade of in a similar way to the first example:


        [TestCase(0)]
        [TestCase(1)]
        [TestCase(2)]
        [TestCase(3)]
        [TestCase(4)]
        public void DataCanBeUpgraded(int versionNumber)
        {            
            ....

With DRY (Don’t Repeat Yourself) in mind, I wanted to ensure that whenever a member of the team updated the schema version number, a new test case would automatically be added. After all, the DRY principle states:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Having to both update the version number in the production code and the value in the TestCase attributes would violate this principle. Unit testing has been likened to Double Entry Bookkeeping and you could argue that keeping the TestCases and the version in sync is a form of that.
But this test has no interest in verifying the current data version. We can do that explicitly in a unit test for the version number itself if we really want.

The single, unambiguous representation in this case belongs in the production code. The test code can make use of it, but we still need a way to generate those test cases.

Enter TestCaseSource

It turns out that there is such a way provided by NUnit – the TestCaseSource attribute. It allows you to specify a static method that returns an IEnumerable containing your test case parameters as shown below.


    [TestCaseSource("GetTestCaseInputs")]
    public void APlusBEqualsC(int a, int b, int c)
    {
        Assert.AreEqual(c, a + b, $"Expected {a} + {b} = {c}");
    }

    public static IEnumerable GetTestCaseInputs()
    {
        return new []
        {
            new [] { 1, 1, 2 },
            new [] { 1, 2, 3 },
            new [] { 2, 2, 4 },
        };
    }

Even better, with C#6 I can use the nameof keyword to turn that “GetTestCaseInputs” magic string into a direct reference to the method itself that will be validated at compile-time:


    [TestCaseSource(nameof(GetTestCaseInputs)]

So back to my use-case. To generate test cases for upgrade tests from versions 0-n, I can simply use the Enumerable.Range method with the version number to supply those inputs:


    [TestCaseSource(nameof(GetSupportedVersions))]
    public void DataCanBeUpgraded(int versionNumber)
    {    
         ....

    public static IEnumerable GetSupportedVersions()
    {
        return Enumerable.Range(0, Constants.CurrentVersionNumber);
    }

And now every time someone increments the version number, a new test case magically appears!

Making it obvious what has happened

We’re not necessarily done yet. In my specific case, let’s assume another team member falls foul of changing the version without considering the tests and adding new test data. It’s not fair to leave them floundering around with some arbitrary error message that just magically popped up in a test that didn’t exist two minutes ago. It’s important to be explicit and tell them what they need to do next:

    if (!dataLoaded)
    {
Assert.Fail($"Could not find test data to load for version {versionNumber}.Did you increase the current version? If so, you need to add sample test data.");
    } 

Check out the full example code on github.

A future proof solution

Great! So we now have automatically generated test cases, we have avoided violating the DRY principle and we have an explicit message for anyone who forgot to add new test data. Hopefully this will prove to be a powerful safety net and to some extent – a future proof solution!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s