aspnetcore

Code coverage with Coverlet in MSBuild and Azure Pipelines

I recently started using Coverlet for code coverage in some projects, the projects are all hosted on GitHub or Azure DevOps and build using MSBuild and Azure Pipelines. In this post I will describe how I'm using it.

What is Coverlet

Coverlet is a cross platform code coverage framework for .NET, with support for line, branch and method coverage. It works with .NET Framework on Windows and .NET Core on all supported platforms.

MSBuild Integration

Coverlet also integrates with the build system to run code coverage after tests. Enabling code coverage is as simple as setting the CollectCoverage property to true.

> dotnet test /p:CollectCoverage=true

Add Coverlet to the test projects

To add Coverlet to your test project use the following command.

> dotnet add package coverlet.msbuild

Directory.Build.props

You can also add it to all of your test projects at once by adding a Directory.Build.props file to your tests folder. Using Directory.Build.props files you can add a new property to every project in one step by defining it in the root folder that contains your source. When MSBuild runs, Microsoft.Common.props searches your directory structure for the Directory.Build.props file (and Microsoft.Common.targets looks for Directory.Build.targets). If it finds one, it imports the property.

<Project>
    <ItemGroup>
        <PackageReference Include="coverlet.msbuild" Version="2.6.3">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
    </ItemGroup>
</Project>

Coverage output

Coverlet can generate coverage results in multiple formats, which is specified using the CoverletOutputFormat property. For example, the following command emits coverage results in the cobertura format.

> dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

Include and/or exclude code

You can ignore a method or an entire class from code coverage by creating and applying the ExcludeFromCodeCoverage attribute present in the System.Diagnostics.CodeAnalysis namespace.

Coverlet also gives you the ability to have fine grained control over what gets excluded or included using "filter expressions". In the following example we include all projects that start with CompanyName., and exclude all projects that end with *Tests.

> dotnet test /p:CollectCoverage=true /p:Include="[CompanyName.*]*" /p:Exclude="[*Tests]*"

Note: To exclude or include multiple assemblies when using Powershell scripts or creating a .yaml file for a Azure DevOps build %2c should be used as a separator. MSBuild will translate this symbol to ,.

Azure Pipelines

In a Azure Pipeline you can use the .NET Core CLI task to run your tests. You have to add some extra arguments to enable the coverage.

Configure the test to collect coverage

To run the code coverage in our Azure Pipeline we need to configure the .NET Core CLI task. We can use the arguments from the dotnet test command.

- task: DotNetCoreCLI@2
  displayName: Test
  inputs:
    command: test
    projects: |
      **/tests/*.Tests/*.csproj
    arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Include="[ProjectName.*]*" /p:Exclude="[*Tests]*"'

Generate the code coverage report

To generate the code coverage report from the individual code coverage results per test project, you can use the ReportGenerator task. ReportGenerator converts coverage reports generated by a number of reporters into human readable reports in various formats.

- task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
  displayName: Generate Code Coverage Report
  inputs:
    reports: $(Build.SourcesDirectory)/tests/**/coverage.cobertura.xml
    targetdir: $(build.artifactstagingdirectory)/TestResults/
    reporttypes: 'HtmlInline_AzurePipelines;Cobertura;Badges'

Publish the code coverage results

Publish the code coverage results with the Publish Code Coverage Results task

- task: PublishCodeCoverageResults@1
  displayName: 'Publish Code Coverage Results'
  inputs:
    codeCoverageTool: cobertura
    summaryFileLocation: $(build.artifactstagingdirectory)/TestResults/cobertura.xml
    # To make the task not regenerate the report an environment variable was added to the pipeline in Azure DevOps; "disable.coverage.autogenerate: 'true'"
    # see: https://github.com/danielpalme/ReportGenerator/wiki/Integration#attention
    reportDirectory: '$(build.artifactstagingdirectory)/TestResults'

The Publish Code Coverage Results task from Microsoft regenerates the report with different settings and based on the supplied Coberatura file. Moreover it does not necessarily use the latest version of ReportGenerator. To disable the regeneration of the report, you need to use the following environment variable in your build (in Azure DevOps).

disable.coverage.autogenerate: 'true'

Environment variable

Build time SASS compiling, and bundling and minifying of CSS and JS using Gulp

In this post, I will describe how to use Gulp tasks to automate the compiling of SASS files and the bundling and minifying of CSS and JS files inside a Razor class library (RCL). The goal is to run the tasks on every build using the dotnet build or when building using Visual Studio to make it as seamless as possible.

For our example we will be compiling a SASS file site.scss and bundling a couple of JS files site.js, lib-a.js and lib-b.js. These files are located in the wwwroot folder of our RCL project.

The folder structure looks like this:

wwwroot
    | theme
        | css
        | js
        - site.scss
        - site.js
        - lib-a.js
        - lib-b.js

The root of the theme folder contains the files we need to compile, bundle and minify. And the css and js folders will contain the resulting files.

Gulp tasks

I won't go into much detail on the actual Gulp tasks, because that is a whole other topic. But the following is the basic setup needed for our example.

The package.json used to install the dependencies needed for the tasks.

{
  "private": true,
  "devDependencies": {
    "gulp": "3.9.1",
    "gulp-concat": "2.6.1",
    "gulp-cssmin": "0.2.0",
    "gulp-uglify": "3.0.2",
    "rimraf": "2.6.3",
    "gulp-sass": "4.0.2",
    "run-sequence": "2.2.1"
  }
}

The gulpfile.js with the tasks to compile the SASS (sass) and a task for bundling and minifying the CSS and JS (min). And a task to clean the folders (clean).

var gulp = require('gulp'),
    rimraf = require('rimraf'),
    concat = require('gulp-concat'),
    cssmin = require('gulp-cssmin'),
    uglify = require('gulp-uglify'),
    sass = require('gulp-sass'),
    runSequence = require('run-sequence');

var paths = {
    root: './wwwroot/theme/'
};

paths.js = paths.root + '*.js';
paths.minJs = paths.root + 'js/*.min.js';
paths.css = paths.root + 'css/*.css';
paths.minCss = paths.root + 'css/*.min.css';
paths.concatJsDest = paths.root + 'js/site.min.js';
paths.concatCssDest = paths.root + 'css/site.min.css';

gulp.task('default', function (done) {
    runSequence('clean', 'sass', 'min', function () { done(); });
});

gulp.task('clean:js', function (cb) {
    rimraf(paths.concatJsDest, cb);
});
gulp.task('clean:css', function (cb) {
    rimraf(paths.concatCssDest, cb);
});
gulp.task('clean', ['clean:js', 'clean:css']);

gulp.task('sass', function () {
    return gulp.src(paths.root + '/site.scss')
        .pipe(sass())
        .pipe(gulp.dest(paths.root + '/css'));
});

gulp.task('min:js', function () {
    return gulp.src([paths.js, '!' + paths.minJs], { base: '.' })
        .pipe(concat(paths.concatJsDest))
        .pipe(uglify())
        .pipe(gulp.dest('.'));
});
gulp.task('min:css', function () {
    return gulp.src([paths.css, '!' + paths.minCss])
        .pipe(concat(paths.concatCssDest))
        .pipe(cssmin())
        .pipe(gulp.dest('.'));
});
gulp.task('min', ['min:js', 'min:css']);

Execute the tasks from a project file

To execute the Gulp task during the build process we need add some build tasks to the *.csproj file.

To execute a task in a project file, create an Exec element with the command for the task as a child of a Target element. And specify that this target should run before the Build target.

<Target Name="MyPreCompileTarget" BeforeTargets="Build"> ... </Target>

Ensure Node.js is installed

Because Gulp depends on Node.js we need to ensure that it is installed before we run the tasks. We can do that by adding a command that checks the installed version of node, and if that command fails we throw an error to indicate that it needs to be installed.

<Exec Command="node --version" ContinueOnError="true">
    <Output TaskParameter="ExitCode" PropertyName="ErrorCode" />
</Exec>
<Error Condition="'$(ErrorCode)' != '0'" Text="Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE." />

Building the project without having Node.js installed will now give you a nice error message.

11>..\MyRazorUI\MyRazorUI.csproj(18,9): error : Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE.

Restoring dependencies using NPM

If Node.js is installed we can restore the dependencies needed to run the Gulp task.

<Message Importance="high" Text="Restoring dependencies using 'npm'. This may take several minutes..." />
<Exec WorkingDirectory="$(ProjectDir)" Command="npm install" />

Run the Gulp tasks

When everything has been installed en restored, we can run the tasks to compile the SASS files and bundle and minify the CSS and JS files.

<Exec WorkingDirectory="$(ProjectDir)" Command="node_modules\.bin\gulp default" />

The completed Target element looks like this:

<Target Name="MyPreCompileTarget" BeforeTargets="Build">
    <!-- Ensure Node.js is installed -->
    <Exec Command="node --version" ContinueOnError="true">
        <Output TaskParameter="ExitCode" PropertyName="ErrorCode" />
    </Exec>
    <Error Condition="'$(ErrorCode)' != '0'" Text="Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE." />
    <Message Importance="high" Text="Restoring dependencies using 'npm'. This may take several minutes..." />
    <Exec WorkingDirectory="$(ProjectDir)" Command="npm install" />
    <Exec WorkingDirectory="$(ProjectDir)" Command="node_modules\.bin\gulp default" />
</Target>

Include the bundled and minified files in a Razor class library (RCL)

What is a Razor class library (RCL)

Razor views, pages, controllers, page models, Razor components, View components, and data models can be built into a RCL. This RCL can be packaged and reused.

See Razor class libraries for more information. Razor class libraries requires .NET Core 2.1 SDK or later.

Embedding static files in a Razor class library

We want our bundled and minified files also to be packaged in our RCL. But Razor class libraries by default can not expose static files. For this you need to embed your static assets into your RCL assembly and add a FileProvider to serve your static files.

For embedding the files we can use a wildcard include to include multiple files at once. And we don't need (or want) to package the sources, so we won't include them.

<ItemGroup>
    <EmbeddedResource Include="wwwroot\**\css\*" />
    <EmbeddedResource Include="wwwroot\**\js\*" />
</ItemGroup>

Serving static files

To actually be able to serve the embedded files in our application we need to create an additional FileProvider in our RCL, pointing to the resources folder, and adds it to those that retrieve static files. This will allow you to reference the files in your HTML like any other static file.

public class StaticFilePostConfigureOptions : IPostConfigureOptions<StaticFileOptions>
{
    private readonly IHostingEnvironment _environment;

    public StaticFilePostConfigureOptions(IHostingEnvironment environment)
    {
        _environment = environment;
    }

    public void PostConfigure(string name, StaticFileOptions options)
    {
        options.ContentTypeProvider = options.ContentTypeProvider ?? new FileExtensionContentTypeProvider();
        if (options.FileProvider == null && _environment.WebRootFileProvider == null)
            throw new InvalidOperationException("Missing FileProvider.");

        options.FileProvider = options.FileProvider ?? _environment.WebRootFileProvider;

        var filesProvider = new ManifestEmbeddedFileProvider(GetType().Assembly, "wwwroot");
        options.FileProvider = new CompositeFileProvider(options.FileProvider, filesProvider);
    }
}

Make sure we configure the FileProvider in the dependency injection container of our application.

services.ConfigureOptions(typeof(StaticFilePostConfigureOptions));

And because the FileProvider needs manifest of the embedded files, we need to generate that manifest too. For this we set the GenerateEmbeddedFilesManifest property in the project file to true.

<PropertyGroup>
    ...
    <GenerateEmbeddedFilesManifest>true</GenerateEmbeddedFilesManifest>
    ...
</PropertyGroup>

Using the compiled, bundled and minified CSS and JS files

Now we can now add a reference to the embedded files in HTML files in any web project that references the RCL.

<script src="~/theme/css/app.bundle.js" asp-append-version="true"></script>
<script src="~/theme/js/site.min.js" asp-append-version="true"></script>

You can see an implementation in Opw.PineBlog.RazorPages. github.com/ofpinewood/pineblog/tree/master/src/Opw.PineBlog.RazorPages

Returning exceptions as ASP.NET Core Problem Details

Problem Details are a machine-readable format for specifying errors in HTTP API responses based on https://tools.ietf.org/html/rfc7807. By providing more specific machine-readable error responses, the API clients can react to errors more effectively and it also makes the APIs much more reliable from the REST API testing perspective and the clients as well.

Some background

For years I have been building WebAPIs for different projects and every time I have written similar code, or copied and updated code form a previous project, to return exceptions over HTTP in a uniform way.
In the end of 2016 I updated my code to return RFC 7807 Problem Details for HTTP APIs compatible responses. And then with the release of ASP.NET Core 2.1 when Problem Details where introduced into the framework I updated the code to use the default Microsoft.AspNetCore.Mvc.ProblemDetails.
When I recently started a new project and I was doing that same implementation again for the umpteenth time, I decided to make a package out of it. And I thought it would be fun to create an OSS project, my first one ;)

The project consists of two parts HttpExceptions and extensions for returning exceptions as ASP.NET Core Problem Details. Both can be found on GitHub at: github.com/ofpinewood/http-exceptions.

Opw.HttpExceptions.AspNetCore

This packages has extensions for returning exceptions as ASP.NET Core Problem Details.

NuGet Badge

Opw.HttpExceptions

This package contains HTTP-specific exception classes that enable ASP.NET to generate exception information. These classes can be used by themselves or as base classes for your own HttpExceptions.

NuGet Badge

Getting started

Add the HttpExceptions services and the middleware in the Startup.cs of your application. First add the required services to the services collection.

public void ConfigureServices(IServiceCollection services)
{
    ...
    services.AddHttpExceptions();
    ...
}

Then you can add the HttpExceptions middleware using the application builder. UseHttpExceptions should be the first middleware component added to the pipeline. That way the UseHttpExceptions Middleware catches any exceptions that occur in later calls. When using HttpExceptions you don't need to use UseExceptionHandler or UseDeveloperExceptionPage.

public void Configure(IApplicationBuilder app)
{
    app.UseHttpExceptions(); // this is the first middleware component added to the pipeline
    ...
}

Configuring the HttpExceptions middleware

You can extend or override the default behavior through the configuration options, HttpExceptionsOptions.

Include exception details

Whether or not to include the full exception details in the response. The default behavior is only to include exception details in a development environment.

services.AddHttpExceptions(options =>
{
    // This is the same as the default behavior; only include exception details in a development environment.
    options.IncludeExceptionDetails = context => context.RequestServices.GetRequiredService<IHostingEnvironment>().IsDevelopment();
});

Is Exception Response

Is the response an exception and should it be handled by the HttpExceptions middleware.

services.AddHttpExceptions(options =>
{
    // This is a simplified version of the default behavior; only include exception details for 4xx and 5xx responses.
    options.IsExceptionResponse = context => (context.Response.StatusCode < 400 && context.Response.StatusCode >= 600);
});

Custom ExceptionMappers

Set the ExceptionMapper collection that will be used during mapping. You can override and/or add ExceptionMappers for specific exception types. The ExceptionMappers are called in order so make sure you add them in the right order.

By default there is one ExceptionMapper configured, that ExceptionMapper catches all exceptions.

services.AddHttpExceptions(options =>
{
    // Override and or add ExceptionMapper for specific exception types, the default ExceptionMapper catches all exceptions.
    options.ExceptionMapper<BadRequestException, BadRequestExceptionMapper>();
    options.ExceptionMapper<ArgumentException, ExceptionMapper<ArgumentException>>();
    // The last ExceptionMapper should be a catch all, for type Exception.
    options.ExceptionMapper<Exception, MyCustomExceptionMapper>();
});

Where can I get it?

The code can be found on GitHub at: github.com/ofpinewood/http-exceptions. And there is also a sample project you can have a look at.

You can install the Opw.HttpExceptions and Opw.HttpExceptions.AspNetCore NuGet packages from the package manager console:

PM> Install-Package Opw.HttpExceptions
PM> Install-Package Opw.HttpExceptions.AspNetCore