ASP.NET Core Application Testing With SpecFlow

aspnetcore, webdev, testing, dotnetcore, dev, devops

By Arnaud , 10 April 2017

I started a series a few weeks ago about testing ASP.NET Core applications using a different approach: the Behavior Approach. I am very convinced of this method and very excited about presenting you its different facets. To get more details about this approach, please take a look at this post.

The behavior approach does not intend to test a single unit of code but rather a global scenario with the aim of validating a functionnal case.

C# is not the best way to describe a behavior test because of its technical context. A language has been designed for this purpose: Gherkin and its .NET implementation: Specflow.

Specflow diagram from specflow.org

Presentation of Specflow

Here is what a simple behavior test could look like with Specflow:

Feature: Get blogs
    In order to list the available blogs
    I want to be told the list of blogs

Background:
    Given A configured environment

Scenario: Get blogs should be ok
    Given the following blogs
    | Url             |
    | http://blog1.io |
    | http://blog2.io |
    When I get the list of blogs
    Then the result must be the following list
    | Url             |
    | http://blog1.io |
    | http://blog2.io |

Very easy to read and to understand especially from a non-technical point of view. SpecFlow provides the ability to generate some .NET code from a Feature (like the previous one "Get blogs") targeting a specific Unit Test framework (the default one is MSTest).
The main idea is to compose a Feature with Scenarios and to associate each sentence of the scenario to a Step that will be invoked. Here is the C# representation of the steps described into the scenario:

[Binding]
public class MainSteps
{
    [Given(@"A configured environment")]
    public void GivenAWorkingEnvironment()
    {
        // Configure the test environment
    }

    [Given(@"the following blogs")]
    public void GivenTheFollowingBlogs(Table table)
    {
        // Configure the test data context
    }

    [When(@"I get the list of blogs from Api")]
    public void WhenIGetTheListOfBlogs()
    {
        // Make the call
    }

    [Then(@"the result must be the following list")]
    public void ThenTheResultMustBeTheFollowingList(Table expectedResult)
    {
        // Assert the result
    }
}

These steps are then translated by SpecFlow to the targeted test framework code. For MSTest, each Scenario willl generate a TestMethod method within a TestClass class and so make the test discoverable by the test runner!
For more information about Specflow, please take a look at its documentation.

Integration with ASP.NET Core

Unfortunatly, at the time of writing this article, there is no official port of SpecFlow for .NET Core :( but there is an unofficial solution :P
Thanks to statjs, a workaround is available with the SpecFlow.NetCore nuget package. Indeed SpecFlow generates code from Steps by introspecting the associated csproj file. The problem is that SpecFlow is compatible with the legacy csproj format but not with the new one brought with Visual Studio 2017.
As a consequence SpecFlow.NetCore generates a old-school csproj file from the SpecFlow files detected within the project and then run the SpecFlow tool that will generate the testable code.

Configuration

Let's see how to configure a test project to test .NET Core code from SpecFlow in Visual Studio 2017.

• Create a new .NET Core Unit Test Project (or xUnit Test Project if want to use xUnit).
• Edit the csproj.
• Add the following package references:

<ItemGroup>
   ...
   <PackageReference Include="SpecFlow" Version="2.1.0" />
   <PackageReference Include="SpecFlow.NetCore" Version="1.0.0-rc8" />

We now have to launch the SpecFlow.NetCore tool before compilation to generate the test code that will be discovered by the test runner after compilation:

• Add a .Net CLI tool reference to SpecFlow.NetCore

<ItemGroup>
  <DotNetCliToolReference Include="SpecFlow.NetCore" Version="1.0.0-rc8" />
</ItemGroup>

• Configure the tool to be executed before the build

<Target Name="PrecompileScript" BeforeTargets="BeforeBuild">
  <Exec Command="dotnet SpecFlow.NetCore" />
</Target>

.NET Multi-targeting

The SpecFlow official package provides the object model required to configure a feature that will be translated to tests.

The SpecFlow nuget package targets the full .NET framework. To be able to reference both the full .NET framework (for SpecFlow) and .NET Core (for the ASP.NET Core application we want to test), we will have to use a multi-targeting approach:

Indeed, we have to configure our ASP.NET Core application so it targets .NET Core and the .NET framework in order to be referenced by our Unit Test Project (which reference the SpecFlow package and so will need to target the .NET framework).
Edit your ASP.NET Core application csproj file and update the Target Framework Moniker to target also the .NET framework:

<PropertyGroup>
  <TargetFrameworks>netcoreapp1.1;net461</TargetFrameworks>    
</PropertyGroup>

Please note that we use the TargetFrameworks command with a 's'.

Go back then to your Unit Test Project and update the PropertyGroup element containing the Target Framework Moniker:

<PropertyGroup>
  <TargetFrameworks>net461</TargetFrameworks>
  <RuntimeIdentifier>win7-x86</RuntimeIdentifier>
  <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
  <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
</PropertyGroup>

Here we target the version 4.6.1 of the .NET framework and we set a runtime identifier (required when targeting the .NET full framework).
We also activate 2 features GenerateBindingRedirectsOutputType and AutoGenerateBindingRedirects to bypass a bug when referencing Microsoft.AspNetCore.TestHost from .NET Core app targeting the .NET full framework (see this link for more information).

Test Your Features

Now you can test your features with a working SpecFlow configuration for .NET Core !
Define your specification within a .feature file. After compilation, a .feature.cs will be generated containing the discoverable test code. You just have then to write the code associated to the steps in a separated files, I suggest you to create a .steps.cs file next the .feature file.
Ex:

UnitTestProject
   MyFeature/
      MyFeature.feature
      MyFeature.feature.cs
      MyFeature.steps.cs

To help you write your behavior test. Don't forget to take a look to my others posts about Behavior Testing with ASP.NET Core ;):

• A Different Approach To Test Your ASP.NET Core Application
• How To Test Your Asp.net Core MVC Stack
• Asp.Net Core MVC Testing And The Synchronizer Token Pattern
• How To Deal With Identity When Testing An ASP.NET Core Application
• TESTAVIOR: A Happy Solution To Test Your ASP.NET Core Applications