At my current job, one big topic of conversation lately has been how we are passing in our configuration values to our Integration tests. I want to show you how you can use the *.runsettings file to inject in configuration values to our NUnit tests, make your tests more configurable, and make handling configuration values in your tests easier to manage.

Our current solution is not easy to manage and not scalable with the numerous new developers coming onboard and increasing number of tests that we are writing. We have a static configuration class that uses the System.Configuration library to access app.settings section inside the app.config for our test assembly. The idea of this is great, you have central authority for getting configuration values that your tests need. The implementation and execution of this idea is what is holding us back.

In our implementation, we use a python script on our build server in our CI pipeline, that takes in the parameters, such as: The name of dynamically spun up Server Under Test, the WebAPIUrl of our application, the name of the Database, etc. The script then dynamically sets these values by building up the *.dll.config file for the test assembly on the fly based off a template. In order to add a new value to our tests, that are a lot of pieces and components that have to be updated. This makes this solution not scalable and causes numerous headaches when things like binding redirects or other settings in your app.configs for specific Integration tests come up.

How can we change our implementation to make it more configurable and easier to manage? That is where the *.runsettings comes into play. In my previous post about the TestContext, I had briefly mentioned the Parameters member. In this post, I want to dig in a little deeper into this class and how you can best use it in your tests.


First thing we want to do is create our local.runsettings for our local development. All we need to do is add a new file to our project, call it local.runsettings, and add our first parameter. It will end up looking like this:

<RunSettings>  
    <TestRunParameters>
        <Parameter name="WebAppUrl" value="http://localhost" />
    </TestRunParameters>
</RunSettings>  



Now we can add a test that uses this variable. We can make a simple test that just asserts that the value we pull from the Parameters is equal to the value we expect it to be.

[TestFixture]
public class Runsettings  
{
    [Test]
    public void VerifyWebAppUrlParameter()
    {
        string webAppUrl = TestContext.Parameters["WebAppUrl"];
        Assert.That(webAppUrl.Equals("http://localhost"));
    }
}



Now we can run this test to verify that we are pulling back the value from our local.runsettings in our test. Some of you might be ahead of me and saying that this won't pass, and you are right! We still need to do one of two things, depending on where we are running our test, before we can get the value from the local.runsettings and get this test to pass.

If we are running our test in Visual Studio, we need to specify a settings file that the NUnit3 adapter will use when running tests.

Select Runsettings

On your toolbar, go to Test -> Test Settings -> Select Test Settings File. It will open a file browser, where you can then choose the local.runsettings file that we created earlier.

If we are running our test using the NUnit3-Console, we need to pass in the parameter and value in as an argument in the command line.

Without sending in the WebAppUrl value into our command

Command:

D:\SourceCode\nunit-console\bin\Release\nunit3-console.exe .\bin\Debug\BlogSamples.NUnit.dll --test=BlogSamples.NUnit.Runsettings

Output:
D:\SourceCode\BlogSamples\BlogSamples.NUnit> D:\SourceCode\nunit-console\bin\Release\nunit3-console.exe .\bin\Debug\BlogSamples.NUnit.dll --test=BlogSamples.NUnit.Runsettings  
NUnit Console Runner 3.7.0  
Copyright (C) 2017 Charlie Poole

Runtime Environment  
   OS Version: Microsoft Windows NT 10.0.14393.0
  CLR Version: 4.0.30319.42000

Test Files  
    .\bin\Debug\BlogSamples.NUnit.dll

Test Filters  
    Test: BlogSamples.NUnit.Runsettings


Errors, Failures and Warnings

1) Error : BlogSamples.NUnit.Runsettings.VerifyWebAppUrlParameter  
System.NullReferenceException : Object reference not set to an instance of an object.  
   at BlogSamples.NUnit.Runsettings.VerifyWebAppUrlParameter() in D:\SourceCode\BlogSamples\BlogSamples.NUnit\Runsettings.cs:line 29

Run Settings  
    DisposeRunners: True
    WorkDirectory: D:\SourceCode\BlogSamples\BlogSamples.NUnit
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.6.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 12

Test Run Summary  
  Overall result: Failed
  Test Count: 1, Passed: 0, Failed: 1, Warnings: 0, Inconclusive: 0, Skipped: 0
    Failed Tests - Failures: 0, Errors: 1, Invalid: 0
  Start time: 2017-12-07 17:55:14Z
    End time: 2017-12-07 17:55:14Z
    Duration: 0.634 seconds



With sending in WebAppUrl into our command

Command:

D:\SourceCode\nunit-console\bin\Release\nunit3-console.exe .\bin\Debug\BlogSamples.NUnit.dll --test=BlogSamples.NUnit.Runsettings --params WebAppUrl=http://localhost

Output:
D:\SourceCode\BlogSamples\BlogSamples.NUnit> D:\SourceCode\nunit-console\bin\Release\nunit3-console.exe .\bin\Debug\BlogSamples.NUnit.dll --test=BlogSamples.NUnit.Runsettings --params WebAppUrl=http://localhost  
NUnit Console Runner 3.7.0  
Copyright (C) 2017 Charlie Poole

Runtime Environment  
   OS Version: Microsoft Windows NT 10.0.14393.0
  CLR Version: 4.0.30319.42000

Test Files  
    .\bin\Debug\BlogSamples.NUnit.dll

Test Filters  
    Test: BlogSamples.NUnit.Runsettings


Run Settings  
    DisposeRunners: True
    WorkDirectory: D:\SourceCode\BlogSamples\BlogSamples.NUnit
    TestParametersDictionary: [WebAppUrl, http://localhost]
    TestParameters: WebAppUrl=http://localhost
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.6.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 12

Test Run Summary  
  Overall result: Passed
  Test Count: 1, Passed: 1, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2017-12-07 17:55:18Z
    End time: 2017-12-07 17:55:19Z
    Duration: 0.622 seconds



It's important to note the behavior when you try accessing the Parameter member and asking for a key that isn't in the *.runsettings file specified or passed into the commandline.

It will throw everyone's favorite NullReferenceException: Object reference not set to an instance of an object. Which is not helpful to us when we are trying to figure out why our tests are failing.

This is where having that separate static configuration class comes in handy. Instead of accessing our parameters directly from the TestContext every time, we can use our configuration class which gives us the ability to check the existence of the parameters before we access them.

This might look something like this.

public class Configuration  
{
    private static string _webAppUrl;
    public static string WebAppUrl
    {
        get
        {
            if (_webAppUrl == null && TestContext.Parameters.Exists("WebAppUrl"))
            {
                _webAppUrl = TestContext.Parameters["WebAppUrl"];
            }
            else
            {
                throw new System.ArgumentException($"The parameter 'WebAppUrl' was not found, please provide a value for this parameter.");
            }
            return _webAppUrl;
        }
    }
}

If we move the accessor into our configuration class, we can then update our test to look like this instead.

[Test]
public void VerifyWebAppUrlParameter()  
{
    Assert.That(Configuration.WebAppUrl.Equals("http://localhost"));
}

Now tooled with this implementation, we can:

  1. Remove the need for dynamically creating these *.dll.config files on the fly.
  2. Allow people to add new configuration values with more ease.
  3. Pull out our configuration values from our tests for more dynamic execution.

For us, our solution becomes extremely simplified. We can still use our static configuration class, and how we access our configuration values does not change. The big change for us is completely removing the need for building up the *.dll.config and just pass those parameters into the NUnit3-Console when executing our tests making our process much simpler and easier to manage!

I am curious to hear how you all pass in your configuration values to your tests. Maybe there is a better way yet, but for now, this is the best solution that I have found!

© 2018. All Rights Reserved.

Proudly published with Ghost