Startup Configuration (like ASP.NET)

[SabotageAndi]

Idea from https://github.com/techtalk/SpecFlow/issues/1158

From @ChristopherHaws:

I really like this approach because there have been times where I have wanted to hook into specflow in areas that only plugins can hook in. This would allow me to do that without the need to create a plugin. For example:

public class Hooks
{
    [SpecFlowStartup]
    public static void Startup(ITestHost testHost)
    {
        testHost.Events.RegisterGlobalDependencies(container =>
        {
            // Register global dependencies here...
        });
    }
}

From @Tragedian

public class SpecFlowStartup
{
   public void Configure(ITestHostBuilder testHost)
   {
      testHost.Plugins.Insert(2, new MyPluginDescription());
   }
}

I am shamelessly stealing from the Startup class concept from the OWIN/ASP.NET application startup routines. I love this convention-based pattern and having my own SpecFlowStartup class as a way to configure my test runner would be dreamy. No attributes; just explicit configuration in code.

[ChristopherHaws]

I would really like this feature to become a first class citizen in SpecFlow. Currently, we use a runtime plugin to fake this behavior, but it has some weird behaviors. Obviously the code below only supports modifying the IoC container, but I could see this growing to support a whole host of useful features.

public abstract class SpecFlowStartup
{
    public virtual void Configure(SpecFlowConfiguration configuration) { }
    public virtual void ConfigureEarlyGlobalServices(IObjectContainer container) { }
    public virtual void ConfigureGlobalServices(IObjectContainer container) { }
    public virtual void ConfigureTestThreadServices(IObjectContainer container) { }
    public virtual void ConfigureScenarioServices(IObjectContainer container) { }
}

public class StartupPlugin : IRuntimePlugin
{
    private static SpecFlowStartup Instance;
    private static readonly Object SyncObject = new Object();

    public void Initialize(RuntimePluginEvents runtimePluginEvents, RuntimePluginParameters runtimePluginParameters)
    {
        if (String.IsNullOrWhiteSpace(runtimePluginParameters.Parameters))
        {
            throw new SpecFlowException($@"The startup plugin requires a type name to be set as the parameter. (i.e. <add name=""SpecFlow.Startup"" parameters=""ProjectName.Startup, ProjectName"" />)");
        }

        // Initialize gets called once per test thread, but we only want to set this up once when in SharedAppDomain mode...
        lock (SyncObject)
        {
            if (Instance != null)
            {
                return;
            }

            var type = Type.GetType(runtimePluginParameters.Parameters, false, false);
            if (type == null)
            {
                throw new SpecFlowException($"Startup type '{runtimePluginParameters.Parameters}' could not be found.");
            }

            Instance = Activator.CreateInstance(type) as SpecFlowStartup;
            if (Instance == null)
            {
                throw new SpecFlowException($"Type '{type.FullName}' does not implement '{nameof(SpecFlowStartup)}'.");
            }

            runtimePluginEvents.ConfigurationDefaults += (sender, args) => Instance.Configure(args.SpecFlowConfiguration);
            runtimePluginEvents.RegisterGlobalDependencies += (sender, args) => Instance.ConfigureEarlyGlobalServices(args.ObjectContainer);
            runtimePluginEvents.CustomizeGlobalDependencies += (sender, args) => Instance.ConfigureGlobalServices(args.ObjectContainer);
            runtimePluginEvents.CustomizeTestThreadDependencies += (sender, args) => Instance.ConfigureTestThreadServices(args.ObjectContainer);
            runtimePluginEvents.CustomizeScenarioDependencies += (sender, args) => Instance.ConfigureScenarioServices(args.ObjectContainer);
        }
    }
}

[Code-Grump]

If Startup-based configuration is introduced, how should it factor in with the existing config-file-based configuration? In .NET Core, configuration files are managed by loading the configuration into a common in-memory structure via Startup. This gives significant flexibility to load configuration from a variety of sources, with the downside that configuration must be explicitly loaded.

I think this is the most significant thing to decided: whether a SpecFlowStartup class is the sole source of configuration of SpecFlow, or whether it's a complimentary source alongside the established configuration mechanisms.

My preference is to start afresh, follow the .NET Core conventions, and make this the sole way to configure a SpecFlow run. That might mean adding some backwards-compatibility shims that let you configure SpecFlow from the established configuration files, but there would be an explicit call for them. Something like a ConfigureFromSpecFlowJsonFile extension method:

public class SpecFlowStartup
{
    public void Configure(ITestHostBuilder testHost)
    {
        testHost.ConfigureFromSpecFlowJsonFile("specflow.json");
    }
}

This would be a point of friction for users upgrading, but I think the explicitness is worth the pain.

[SabotageAndi]

One thing that should be ignored when we are talking about configuration, is that the Visual Studio extension also has to read the configuration (feature language, additional step assemblies, ...).

[Code-Grump]

@SabotageAndi Is there currently any overlap between the tooling settings and the test host settings? It might help steer my thoughts on the matter.

[SabotageAndi]

@Tragedian The feature language is used by syntax highlighting, intellisense and code behind file generation. Additional step assemblies are used by the IntelliSense and the runtime.

That's the 2 config values I have currently in my mind that are shared.

[Code-Grump]

I think that means the configuration file support has to be baked-in like it is today; if you specify additional step assemblies in your app.config or specflow.json file, you would expect them to work without having to configure anything else.

And whilst the tooling could look for and run the same SpecFlowStartup class, it seems far too problematic to try and have a code-generation tool reliant on compiling the assembly it is generating code for.

This means that at the stage when SpecFlowStartup is called by the runtime, it must have already loaded in the configuration files into whatever builder state is being exposed. This has the advantage of somewhat easier backwards compatibility support; the existing setup routine can just be extended to call into the startup class after the existing configuration load has happened. Nobody's workflow should be broken.

[Code-Grump]

But looking at .NET Core and the tooling around it, I can't find any good examples of design-time configuration flowing into runtime configuration.

Could a possible solution be to generate some artefact as part of the tooling which could be consumed by the Startup code? Like have a specflow.stepassemblies.json produced as part of the design-time build and embedded as a resource into the test assembly, or otherwise shipped with the test so it can be read in by the runtime?

Just throwing out ideas; I'd like to see a very clean separation between design-time settings and runtime configuration where at all possible.

[ChristopherHaws]

@Tragedian One example of design time merging with runtime is Entity Framework Core. Anytime you do a pretty much anything with the CLI, it actually invokes your host builder, grabs the services, and attempts to locate the DbContext in which you are running a command against. If you don't have a host builder or you want to do more complicated things, they use an interface that you implement IDesignTimeDbContextFactory. I am not saying that this is great, but it does work.

https://docs.microsoft.com/en-us/ef/core/miscellaneous/cli/dbcontext-creation https://docs.microsoft.com/en-us/ef/core/miscellaneous/cli/services

[Code-Grump]

It'd be interesting to see how they do that. It still sounds very complicated, but might be a good replacement for config files.