Quartz 4

119_Managing_Message_Storage

5 min read

Managing Message Storage

::: info Wolverine will automatically check for the existence of necessary database tables and functions to support the configured message storage, and will also apply any necessary database changes to comply with the configuration automatically. :::

Wolverine uses the Oakton “Stateful Resource” model for managing infrastructure configuration at development or even deployment time for configured items like the database-backed message storage or message broker queues.

Disable Automatic Storage Migration

To disable the automatic storage migration, just flip this flag:

using var host = await Host.CreateDefaultBuilder()
    .UseWolverine(opts =>
    {
        // Disable automatic database migrations for message
        // storage
        opts.AutoBuildMessageStorageOnStartup = AutoCreate.None;
    }).StartAsync();

snippet source | anchor

Programmatic Management

Especially in automated tests, you may want to programmatically rebuild or clear out all persisted messages. Here’s a sample of the functionality in Wolverine to do just that:

// IHost would be your application in a testing harness
public static async Task testing_setup_or_teardown(IHost host)
{
    // Programmatically apply any outstanding message store
    // database changes
    await host.SetupResources();
 
    // Teardown the database message storage
    await host.TeardownResources();
 
    // Clear out any database message storage
    // also tries to clear out any messages held
    // by message brokers connected to your Wolverine app
    await host.ResetResourceState();
 
    var store = host.Services.GetRequiredService<IMessageStore>();
 
    // Rebuild the database schema objects
    // and delete existing message data
    // This is good for testing
    await store.Admin.RebuildAsync();
 
    // Remove all persisted messages
    await store.Admin.ClearAllAsync();
}

snippet source | anchor

Building Storage on Startup

To have any missing database schema objects built as needed on application startup, just add this option:

// This is rebuilding the persistent storage database schema on startup
builder.Host.UseResourceSetupOnStartup();

snippet source | anchor

Command Line Management

Assuming that you are using Oakton as your command line parser in your Wolverine application as shown in this last line of a .NET 6/7 Program code file:

// Opt into using JasperFx for command parsing
await app.RunJasperFxCommands(args);

snippet source | anchor

And you’re using the message persistence from either the WolverineFx.SqlServer or WolverineFx.Postgresql or WolverineFx.Marten Nugets installed in your application, you will have some extended command line options that you can discover from typing dotnet run -- help at the command line at the root of your project:

The available commands are:
 
  Alias       Description
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  check-env   Execute all environment checks against the application
  codegen     Utilities for working with JasperFx.CodeGeneration and JasperFx.RuntimeCompiler
  db-apply    Applies all outstanding changes to the database(s) based on the current configuration
  db-assert   Assert that the existing database(s) matches the current configuration
  db-dump     Dumps the entire DDL for the configured Marten database
  db-patch    Evaluates the current configuration against the database and writes a patch and drop file if there are
              any differences
  describe    Writes out a description of your running application to either the console or a file
  help        List all the available commands
  resources   Check, setup, or teardown stateful resources of this system
  run         Start and run this .Net application
  storage     Administer the envelope storage

There’s admittedly some duplication here with different options coming from Oakton itself, the Weasel.CommandLine library, and the storage command from Wolverine itself. To build out the schema objects for message persistence, you can use this command to apply any outstanding database changes necessary to bring the database schema to the Wolverine configuration:

dotnet run -- db-apply

::: info The db-apply, db-assert, db-patch, db-dump, and db-list commands come from Weasel. See the per-command references at:

  • db-apply — apply all outstanding changes to the configured database(s)
  • db-assert — assert the live schema matches the configuration (good for CI deploy gates)
  • db-patch — emit a SQL patch + rollback file for pending changes
  • db-dump — dump the full DDL for the configured database(s)
  • db-list — list configured databases :::

NOTE: See the Exporting SQL Scripts section down the page for details of applying migrations when integrating with Marten

or this option — but just know that this will also clear out any existing message data:

dotnet run -- storage rebuild

or this option which will also attempt to create Marten database objects or any known Wolverine transport objects like Rabbit MQ / Azure Service Bus / AWS SQS queues:

dotnet run -- resources setup

Clearing Node Ownership

::: warning Don’t use this option in production if any nodes are currently running :::

If you ever have a node crash and need to force any persisted, incoming or outgoing messages to be picked up by another node (this should be automatic anyway, but locks might persist and Wolverine might take a bit to recognize that a node has crashed), you can release the ownership of messages of all persisted nodes by:

dotnet run -- storage release

Deleting Message Data

At any time you can clear out any existing persisted message data with:

dotnet run -- storage clear

Exporting SQL Scripts

If you just want to export the SQL to create the necessary database objects, you can use:

dotnet run -- db-dump export.sql

where export.sql should be a file name.

Marten integration

When integrating with Marten, scripts must be generated seperately for both Marten and Wolverine resources.
Resources are separated into databases and can be listed as below:

dotnet run -- db-list

Graph View