pineblog

Creating a custom ConfigurationProvider for a Entity Framework Core source

ASP.NET Core has a lightweight configuration system that is designed to be highly extensible. It lets you aggregate many configuration values from multiple different sources, and then access those in a strongly typed fashion using the Options pattern.

Using packages in the Microsoft.Extensions.Configuration namespace, you can read configuration from:

  • Azure Key Vault
  • Azure App Configuration
  • Command-line arguments
  • Custom providers (installed or created)
  • Directory files
  • Environment variables
  • In-memory .NET objects
  • Settings files

For PineBlog I wanted some values that can be set in the appsettings.json to be set from the admin UI as well. These values are stored in the database and exposed through a Entity Framework Core (EF Core) DbContext.

In this post I'm going to describe creating a custom configuration provider that uses EF Core. For the sake of simplicity it is a more general description than the implementation in PineBlog.

Default configuration

Web apps based on the ASP.NET Core dotnet new templates call CreateDefaultBuilder when building a host. This provides default configuration for the app, for instance the appsettings.json using the File Configuration Provider. For more information see the official documentation on this topic.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
    WebHost.CreateDefaultBuilder(args)
        .UseStartup<Startup>();;

The ConfigOptions

In this example we will have a ConfigOptions class that maps to the configuration in appsettings.json. And a ConfigEntity class that overrides some of those configuration values from the database.

public class ConfigOptions
{
    public string BackgroundColor { get; set; }
    public int ItemsPerPage { get; set; }
    public bool ShowHeader { get; set; }

    // this property will not be overridden by ConfigEntity
    public string ApiKey { get; set; } 
}

public class ConfigEntity
{
    public string BackgroundColor { get; set; }
    public int ItemsPerPage { get; set; }
    public bool ShowHeader { get; set; }
}

The appsettings.json look like this:

{
  "ConfigOptions": {
    "BackgroundColor": "#ff0000",
    "ItemsPerPage": 2,
    "ShowHeader": true,
    "ApiKey": "h&Ww1vbFP6y2RW7Nx$$Q&&KW"
  }
}

Creating a custom configuration provider

With the basics out of the way, we can now start creating our custom configuration provider.

In order to create a custom provider, you need to implement the IConfigurationProvider and IConfigurationSource interfaces from the Microsoft.Extensions.Configuration.Abstractions package. Or you can use any of the provided base classes to get started.

Configuration source

The IConfigurationSource interface only has one method that needs implementing.

And because we will be using EF Core as a configuration source we need a DbContextOptionsBuilder action to use the DbContext from the configuration provider. We also have a ReloadDelay to avoid triggering a reload before a change is completely saved.

public class ConfigEntityConfigurationSource : IConfigurationSource
{
    public Action<DbContextOptionsBuilder> OptionsAction { get; set; }

    public bool ReloadOnChange { get; set; }

    // Number of milliseconds that reload will wait before calling Load. This helps avoid triggering a reload before a change is completely saved. Default is 500.
    public int ReloadDelay { get; set; } = 500;

    public IConfigurationProvider Build(IConfigurationBuilder builder)
    {
        return new ConfigEntityConfigurationProvider(this);
    }
}

Configuration provider

For our custom configuration provider we use the ConfigurationProvider base class, this is the most basic implementation of IConfigurationProvider. The configuration provider initializes the database when it's empty.

We override the Load method with our custom implementation, this method loads (or reloads) the data for the provider. Here we instantiate the DbContext using the OptionsAction from the configuration source. And we then get a ConfigEntity from the database and set its values to the configuration dictionary of the configuration provider.
And by using the same keys for the configuration values (e.g. ConfigOptions.BackgroundColor) as we used in the appsettings.json we will effectively override those values with the values from the database.

public class ConfigEntityConfigurationProvider : ConfigurationProvider
{
    private readonly ConfigEntityConfigurationSource _source;

    public ConfigEntityConfigurationProvider(ConfigEntityConfigurationSource source)
    {
        _source = source;
    }

    public override void Load()
    {
        var builder = new DbContextOptionsBuilder<EntityDbContext>();
        _source.OptionsAction(builder);

        using (var context = new CustomDbContext(builder.Options))
        {
            context.Database.EnsureCreated();

            var config = context.ConfigEntity.SingleOrDefault();
            if (config == null) return;

            Data = new Dictionary<string, string>();
            Data.Add($"{nameof(ConfigOptions)}.{nameof(ConfigOptions.BackgroundColor)}", config.BackgroundColor);
            Data.Add($"{nameof(ConfigOptions)}.{nameof(ConfigOptions.ItemsPerPage)}", config.ItemsPerPage);
            Data.Add($"{nameof(ConfigOptions)}.{nameof(ConfigOptions.ShowHeader)}", config.ShowHeader);
        }
    }
}

We now have a configuration provider that loads its values from the database. But it will only do this once, when the application is started, and we want it to reload as well when the user updates the ConfigEntity in the database.

Reloading configuration on entity changes

To trigger a reload of the configuration when the ConfigEntity in the database changes, we need to let the configuration provider know that the entity has changed. We'll solve this by triggering an event on a entity change observer class and listening for this event in our configuration provider.

Entity change observer

We create a singleton class that has an EventHandler for the entity changes. Our configuration provider can listen for this event to update the configuration.

I've made this class an old school singleton, since injecting it into the DbContext was a bit complicated. And we use ThreadPool.QueueUserWorkItem offload the invoking of the event to a background thread, so it doesn't block the DbContext.SaveChanges.

public class EntityChangeObserver
{
    public event EventHandler<EntityChangeEventArgs> Changed;

    public void OnChanged(EntityChangeEventArgs e)
    {
        ThreadPool.QueueUserWorkItem((_) => Changed?.Invoke(this, e));
    }

    #region singleton

    private static readonly Lazy<EntityChangeObserver> lazy = new Lazy<EntityChangeObserver>(() => new EntityChangeObserver());

    private EntityChangeObserver() { }

    public static EntityChangeObserver Instance => lazy.Value;

    #endregion singleton
}

Notify the observer

Create a custom (base) class that extends DbContext, and override the SaveChanges methods.

public abstract class EntityDbContext : DbContext
{
    public override int SaveChanges()
    {
        OnEntityChange();
        base.SaveChanges()
    }

    public override async Task<int> SaveChangesAsync(CancellationToken cancellationToken)
    {
        OnEntityChange();
        return await base.SaveChangesAsync(cancellationToken);
    }

    private void OnEntityChange()
    {
        foreach(var entry in ChangeTracker.Entries()
            .Where(i => i.State == EntityState.Modified || i.State == EntityState.Added))
        {
            EntityChangeObserver.Instance.OnChanged(new EntityChangeEventArgs(entry));
        }
    }
}

Trigger a reload

To let the configuration provider know the entity has changed and to trigger a reload, we need to listen for the EntityChangeObserver.Changed event. And because we only want to reload when the ConfigEntity changes, we add a check to see if that is the type of the changed entity.

public ConfigEntityConfigurationProvider(ConfigEntityConfigurationSource source)
{
    _source = source;

    if (_source.ReloadOnChange)
        EntityChangeObserver.Instance.Changed += EntityChangeObserver_Changed;
}

private void EntityChangeObserver_Changed(object sender, EntityChangeEventArgs e)
{
    if (e.Entry.Entity.GetType() != typeof(ConfigEntity))
        return;

    Thread.Sleep(_source.ReloadDelay);
    Load();
}

Add the configuration in the application

And finally in the Program.cs we can now configure the IConfigurationSource.

WebHost.CreateDefaultBuilder(args)
    .UseStartup<Startup>()
    .ConfigureAppConfiguration((hostingContext, config) => {
        config.Add(new ConfigEntityConfigurationSource {
            OptionsAction = o => o.UseInMemoryDatabase("db", new InMemoryDatabaseRoot()),
            ReloadOnChange = true
        });
    });

Our application will now load the ConfigOptions from the appsettings.json, then use the ConfigEntityConfigurationSource to override some of the values (when present in the database). And when the user updates the ConfigEntity in the database the values will be reloaded.

Note: Use IOptionsSnapshot to support reloading options with minimal processing overhead. Options are computed once per request when accessed and cached for the lifetime of the request. See the official documentation on reload configuration data with IOptionsSnapshot.

Introducing PineBlog a new ASP.NET Core blogging engine

PineBlog Introducing PineBlog a new blogging engine, light-weight, open source and written in ASP.NET Core MVC Razor Pages, using Entity Framework Core. It is highly extendable, customizable and easy to integrate in an existing web application.

Why another blogging engine?

I've had a blog website for years (I do still need to move the old posts to this blog), and have been using various blogging engines. But I had some time lately and wanted to try some new things, so I thought lets build my own! When I started I knew there were a few things that I really wanted it to be/have:

  • Super easy installation, basically add a NuGet package and done
  • Modern architecture, I chose to use a Clean Architecture (youtube: Clean Architecture with ASP.NET Core)
  • Light-weight, just a blogging engine nothing more..
  • Write my posts in Markdown

So that is what I've been building :) So if you want to know more about it, please read on..
And I will write a more in depth blog post about Clean Architecture later.

Build Status NuGet Badge License: MIT

Features

  • Markdown post editor
  • File management
  • Light-weight using Razor Pages
  • SEO optimized
  • Open Graph protocol
  • Clean Architecture
  • Entity Framework Core, SQL database
  • Azure Blob Storage, for file storage
  • ..only a blogging engine, nothing else..

What is not included

Because PineBlog is very light-weight it is not a complete website, it needs to be integrated in an existing web application of you need to create a basic web application for it. There are a few things PineBlog depends on, but that it does not provide.

  • Authentication and authorization

Note: The admin pages require that authentication/authorization has been setup in your website, the admin area has a AuthorizeFilter with the default policy set to all pages in that area folder.

Where can I get it?

You can install the Opw.PineBlog metapackage from the console.

> dotnet add package Opw.PineBlog

The Opw.PineBlog metapackage includes the following packages.

  • Opw.PineBlog.EntityFrameworkCore package
    The PineBlog data provider that uses Entity Framework Core.
    NuGet Badge

  • Opw.PineBlog.RazorPages package
    The PineBlog UI using ASP.NET Core MVC Razor Pages.
    NuGet Badge

  • Opw.PineBlog.Core package
    The PineBlog core package. This package is a dependency for Opw.PineBlog.RazorPages and Opw.PineBlog.EntityFrameworkCore.
    NuGet Badge

Getting started

You add the PineBlog services and the RazorPages UI in the Startup.cs of your application.

public void ConfigureServices(IServiceCollection services)
{
    ...
    services.AddPineBlog(Configuration);
    
    services.AddMvc().AddPineBlogRazorPages();
    // or services.AddMvcCore().AddPineBlogRazorPages();
    ...
}

Configuration

A few properties need to be configured before you can run your web application with PineBlog.

{
    "ConnectionStrings": {
        "DefaultConnection": "Server=inMemory; Database=pineblog-db;"
    },
    "PineBlogOptions": {
        "Title": "PineBlog",
        "Description": "A blogging engine based on ASP.NET Core MVC Razor Pages and Entity Framework Core",
        "ItemsPerPage": 5,
        "CreateAndSeedDatabases": true,
        "ConnectionStringName": "DefaultConnection",
        "AzureStorageConnectionString": "UseDevelopmentStorage=true",
        "AzureStorageBlobContainerName": "pineblog",
        "FileBaseUrl": "http://127.0.0.1:10000/devstoreaccount1"
    }
}

Blog layout page

For the Blog area you need to override the _Layout.cshtml for the pages, to do this create a new _Layout.cshtml page in the Areas/Blog/Shared folder. This will make the blog pages use that layout page instead of the one included in the Opw.PineBlog.RazorPages package. In the new page you can set the layout page of your website. Make sure to add the head and script sections.

@{
    Layout = "~/Pages/Shared/_Layout.cshtml";
}
@section head {
    @RenderSection("head", required: false)
}
@section scripts {
    @RenderSection("scripts", required: false)
}
@RenderBody()

Your layout page

PineBlog is dependent on Bootstrap 4.3 and Font Awesome 4.7, so make sure to include them in your layout page and add the necessary files to the wwwroot of your project (see the sample project for an example).

<html>
    <head>
        ...
        <environment include="Development">
            <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,700|Merriweather:700">
            <link rel="stylesheet" href="~/css/bootstrap.css" />
            <link rel="stylesheet" href="~/css/font-awesome.min.css">
        </environment>
        <environment exclude="Development">
            <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,700|Merriweather:700">
            <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css"
                asp-fallback-href="~/css/bootstrap.min.css"
                asp-fallback-test-class="sr-only"
                asp-fallback-test-property="position"
                asp-fallback-test-value="absolute"
                integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T"
                crossorigin="anonymous">
            <link rel="stylesheet" href="~/css/font-awesome.min.css" asp-append-version="true">
        </environment>
        ...
    </head>
    <body>
        ...
        <environment include="Development">
            <script src="~/js/jquery.js"></script>
            <script src="~/js/popper.min.js"></script>
            <script src="~/js/bootstrap.js"></script>
        </environment>
        <environment exclude="Development">
            <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"
                    asp-fallback-src="~/js/jquery.min.js"
                    asp-fallback-test="window.jQuery"
                    integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="
                    crossorigin="anonymous">
            </script>
            <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js"
                    integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1"
                    crossorigin="anonymous">
            </script>
            <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"
                    asp-fallback-src="~/js/bootstrap.min.js"
                    asp-fallback-test="window.jQuery && window.jQuery.fn && window.jQuery.fn.modal"
                    integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM"
                    crossorigin="anonymous">
            </script>
        </environment>
    </body>
</html>

Overriding the UI

You can override any other Razor view you like by following the same steps as described above for the layout page. For an example have a look at the sample project where we override the footer (_Footer.cshtml).

Admin layout page

For the Admin area layout page do the same as you did for the Blog area.

...more

For more information, please check PineBlog on GitHub.

Build time SASS compiling, and bundling and minifying of CSS and JS using Gulp

In this post, I will describe how to use Gulp tasks to automate the compiling of SASS files and the bundling and minifying of CSS and JS files inside a Razor class library (RCL). The goal is to run the tasks on every build using the dotnet build or when building using Visual Studio to make it as seamless as possible.

For our example we will be compiling a SASS file site.scss and bundling a couple of JS files site.js, lib-a.js and lib-b.js. These files are located in the wwwroot folder of our RCL project.

The folder structure looks like this:

wwwroot
    | theme
        | css
        | js
        - site.scss
        - site.js
        - lib-a.js
        - lib-b.js

The root of the theme folder contains the files we need to compile, bundle and minify. And the css and js folders will contain the resulting files.

Gulp tasks

I won't go into much detail on the actual Gulp tasks, because that is a whole other topic. But the following is the basic setup needed for our example.

The package.json used to install the dependencies needed for the tasks.

{
  "private": true,
  "devDependencies": {
    "gulp": "3.9.1",
    "gulp-concat": "2.6.1",
    "gulp-cssmin": "0.2.0",
    "gulp-uglify": "3.0.2",
    "rimraf": "2.6.3",
    "gulp-sass": "4.0.2",
    "run-sequence": "2.2.1"
  }
}

The gulpfile.js with the tasks to compile the SASS (sass) and a task for bundling and minifying the CSS and JS (min). And a task to clean the folders (clean).

var gulp = require('gulp'),
    rimraf = require('rimraf'),
    concat = require('gulp-concat'),
    cssmin = require('gulp-cssmin'),
    uglify = require('gulp-uglify'),
    sass = require('gulp-sass'),
    runSequence = require('run-sequence');

var paths = {
    root: './wwwroot/theme/'
};

paths.js = paths.root + '*.js';
paths.minJs = paths.root + 'js/*.min.js';
paths.css = paths.root + 'css/*.css';
paths.minCss = paths.root + 'css/*.min.css';
paths.concatJsDest = paths.root + 'js/site.min.js';
paths.concatCssDest = paths.root + 'css/site.min.css';

gulp.task('default', function (done) {
    runSequence('clean', 'sass', 'min', function () { done(); });
});

gulp.task('clean:js', function (cb) {
    rimraf(paths.concatJsDest, cb);
});
gulp.task('clean:css', function (cb) {
    rimraf(paths.concatCssDest, cb);
});
gulp.task('clean', ['clean:js', 'clean:css']);

gulp.task('sass', function () {
    return gulp.src(paths.root + '/site.scss')
        .pipe(sass())
        .pipe(gulp.dest(paths.root + '/css'));
});

gulp.task('min:js', function () {
    return gulp.src([paths.js, '!' + paths.minJs], { base: '.' })
        .pipe(concat(paths.concatJsDest))
        .pipe(uglify())
        .pipe(gulp.dest('.'));
});
gulp.task('min:css', function () {
    return gulp.src([paths.css, '!' + paths.minCss])
        .pipe(concat(paths.concatCssDest))
        .pipe(cssmin())
        .pipe(gulp.dest('.'));
});
gulp.task('min', ['min:js', 'min:css']);

Execute the tasks from a project file

To execute the Gulp task during the build process we need add some build tasks to the *.csproj file.

To execute a task in a project file, create an Exec element with the command for the task as a child of a Target element. And specify that this target should run before the Build target.

<Target Name="MyPreCompileTarget" BeforeTargets="Build"> ... </Target>

Ensure Node.js is installed

Because Gulp depends on Node.js we need to ensure that it is installed before we run the tasks. We can do that by adding a command that checks the installed version of node, and if that command fails we throw an error to indicate that it needs to be installed.

<Exec Command="node --version" ContinueOnError="true">
    <Output TaskParameter="ExitCode" PropertyName="ErrorCode" />
</Exec>
<Error Condition="'$(ErrorCode)' != '0'" Text="Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE." />

Building the project without having Node.js installed will now give you a nice error message.

11>..\MyRazorUI\MyRazorUI.csproj(18,9): error : Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE.

Restoring dependencies using NPM

If Node.js is installed we can restore the dependencies needed to run the Gulp task.

<Message Importance="high" Text="Restoring dependencies using 'npm'. This may take several minutes..." />
<Exec WorkingDirectory="$(ProjectDir)" Command="npm install" />

Run the Gulp tasks

When everything has been installed en restored, we can run the tasks to compile the SASS files and bundle and minify the CSS and JS files.

<Exec WorkingDirectory="$(ProjectDir)" Command="node_modules\.bin\gulp default" />

The completed Target element looks like this:

<Target Name="MyPreCompileTarget" BeforeTargets="Build">
    <!-- Ensure Node.js is installed -->
    <Exec Command="node --version" ContinueOnError="true">
        <Output TaskParameter="ExitCode" PropertyName="ErrorCode" />
    </Exec>
    <Error Condition="'$(ErrorCode)' != '0'" Text="Node.js is required to build and run this project. To continue, please install Node.js from https://nodejs.org/, and then restart your command prompt or IDE." />
    <Message Importance="high" Text="Restoring dependencies using 'npm'. This may take several minutes..." />
    <Exec WorkingDirectory="$(ProjectDir)" Command="npm install" />
    <Exec WorkingDirectory="$(ProjectDir)" Command="node_modules\.bin\gulp default" />
</Target>

Include the bundled and minified files in a Razor class library (RCL)

What is a Razor class library (RCL)

Razor views, pages, controllers, page models, Razor components, View components, and data models can be built into a RCL. This RCL can be packaged and reused.

See Razor class libraries for more information. Razor class libraries requires .NET Core 2.1 SDK or later.

Embedding static files in a Razor class library

We want our bundled and minified files also to be packaged in our RCL. But Razor class libraries by default can not expose static files. For this you need to embed your static assets into your RCL assembly and add a FileProvider to serve your static files.

For embedding the files we can use a wildcard include to include multiple files at once. And we don't need (or want) to package the sources, so we won't include them.

<ItemGroup>
    <EmbeddedResource Include="wwwroot\**\css\*" />
    <EmbeddedResource Include="wwwroot\**\js\*" />
</ItemGroup>

Serving static files

To actually be able to serve the embedded files in our application we need to create an additional FileProvider in our RCL, pointing to the resources folder, and adds it to those that retrieve static files. This will allow you to reference the files in your HTML like any other static file.

public class StaticFilePostConfigureOptions : IPostConfigureOptions<StaticFileOptions>
{
    private readonly IHostingEnvironment _environment;

    public StaticFilePostConfigureOptions(IHostingEnvironment environment)
    {
        _environment = environment;
    }

    public void PostConfigure(string name, StaticFileOptions options)
    {
        options.ContentTypeProvider = options.ContentTypeProvider ?? new FileExtensionContentTypeProvider();
        if (options.FileProvider == null && _environment.WebRootFileProvider == null)
            throw new InvalidOperationException("Missing FileProvider.");

        options.FileProvider = options.FileProvider ?? _environment.WebRootFileProvider;

        var filesProvider = new ManifestEmbeddedFileProvider(GetType().Assembly, "wwwroot");
        options.FileProvider = new CompositeFileProvider(options.FileProvider, filesProvider);
    }
}

Make sure we configure the FileProvider in the dependency injection container of our application.

services.ConfigureOptions(typeof(StaticFilePostConfigureOptions));

And because the FileProvider needs manifest of the embedded files, we need to generate that manifest too. For this we set the GenerateEmbeddedFilesManifest property in the project file to true.

<PropertyGroup>
    ...
    <GenerateEmbeddedFilesManifest>true</GenerateEmbeddedFilesManifest>
    ...
</PropertyGroup>

Using the compiled, bundled and minified CSS and JS files

Now we can now add a reference to the embedded files in HTML files in any web project that references the RCL.

<script src="~/theme/css/app.bundle.js" asp-append-version="true"></script>
<script src="~/theme/js/site.min.js" asp-append-version="true"></script>

You can see an implementation in Opw.PineBlog.RazorPages. github.com/ofpinewood/pineblog/tree/master/src/Opw.PineBlog.RazorPages