Integration Testing isolated Functions

[jezzsantos]

I've just migrated my functions from netcore3.1 V3 to net6.0 V4. They are simple functions that use an Azure Storage Queue trigger.

Back in .netcore3.1 I was integration testing my functions (end to end) by creating a custom IHost like this:

 this.host = new HostBuilder()
                .ConfigureWebJobs((context, builder) => new Startup().Configure(new WebJobsBuilderContext
                {
                    ApplicationRootPath = context.HostingEnvironment.ContentRootPath,
                    Configuration = context.Configuration,
                    EnvironmentName = context.HostingEnvironment.EnvironmentName
                }, builder.AddAzureStorage(options => options.MaxDequeueCount = 1)))
                .ConfigureServices((context, services) => { services.AddSingleton<IMyApiClient, MyStubApiClient>(); })
                .Build();

            this.host.Start();

and then I could write a simple test that dropped a message on an Azurite storage queue and see that the function picked it up and delivered it. (In my case it just calls a specific API in my backend).

The point is that these integration tests run the functions in the testing process (xUnit Runner), and I get the benefits of debugging in my tests. (No 3rd party hosts, CLI's or the like)

Now that I have migrated to V4 and .Net6.0, I am struggling to use the same basic test harness, which would look like something like this:

            this.host = new HostBuilder()
                .ConfigureFunctionsWorkerDefaults()
                .ConfigureServices((context, services) =>
                {
                    services.AddSingleton<IMyApiClient, MyStubApiClient>();
                })
                .Build();

            this.host.Start();

Except for the fact that, it seems impossible due to these constraints:

• ConfigureFunctionsWorkerDefaults sets up all the middleware, AND also configures GRPC support (services.AddGrpc()). But at runtime fails, because Grpc fails to initialise, missing a bunch of configurations.
  • Even if I provide the necessary configuration (e.g. Port, Host, WorkerId, RequestId and GrpcMaxMessageLength) the code throws exceptions since the endpoint is not actually present. I have no idea what it does and why we need it.
  • Even if I unregister the FunctionRpcClient and the GrpcWorker from the container (via services.RemoveAll()), it still fails at runtime since there is no registered IWorker in the container anymore that is required downstream by WorkerHostedService.
    • I cannot implement my own IWorker implementation because IWorker is internal.
    • I tried replacing your WorkerHostedService with my own class to remove the dependence on IWorker.
      • Now, if I replace all the Grpc stuff and use my own WorkerHostedService, the functions never get triggered by the presence of items on the queues.
• I cannot just replace ConfigureFunctionsWorkerDefaults() with ConfigureFunctionsWorker() (as suggested here) since we are still missing an implementation of IWorker in the container, and I cannot put my own one in the container because that interface is internal.
  • I tried removing the WorkerHostedService and replacing with my own, to remove the dependence on an IWorker and all that works at runtime, except now my functions are never triggered by messages arriving on the queue.

Like so:

            this.host = new HostBuilder()
                .ConfigureFunctionsWorker((_, _) => { }, _ => { })
                .ConfigureServices((context, services) =>
                {
                    services.RemoveAll<IHostedService>();
                    services.AddSingleton<IHostedService, WorkerHostedService>();
                    services.AddSingleton<IMyApiClient, MyStubApiClient>();
                })
                .Build();

            this.host.Start();

using my own IHostedService:

internal class WorkerHostedService : IHostedService
    {
        public async Task StartAsync(CancellationToken cancellationToken)
        {
            await Task.CompletedTask;
        }

        public Task StopAsync(CancellationToken cancellationToken)
        {
            return Task.CompletedTask;
        }
    }

I don't really understand what is needed to be in place for my functions to be triggered by messages arriving on the queues.

Can you give me a clue as to what is missing?

[fabiocav]

@jezzsantos thank you for raising this. We're reviewing the priority of https://github.com/Azure/azure-functions-dotnet-worker/issues/281 to address the issues you've shared, but in the meantime, we're also looking at potential documentation and guidance on how to properly test with the structure we have today.

Thanks!

[stijnmoreels]

+1 Really need some either docs, or guidance as to what is needed inject/remove as otherwise there's no durable way to test custom middleware (don't want to create dedicated Program files for every configurable options).

[stijnmoreels]

Currently implementing our own FunctionContext (https://github.com/arcus-azure/arcus.webapi/blob/master/src/Arcus.WebApi.Tests.Unit/Logging/Fixture/AzureFunctions/TestFunctionContext.cs) so we can at least unit test it. But even this requires reflection and a deep-dive in the internal code.

[fmichellonet]

Trying to reach the same goal and spawn a new host like in @jezzsantos example, but stuck at the grpc init too.

I've noticed grpc parameters (host, port, workerId, requestId, grpcMaxMessageLength) are read from commandline arguments here ; it seems they are provided by the calling process (donet). How are we suppose to pass valid parameters? Any guidance would be highly appreciated.

[dfebresco1]

Any news about this? Same problem here.

[gerocha]

i've just steped in this issue now

[Ciaanh]

With the release of .Net 8 this issue is the main pain point preventing us from migrating our in-process functions to isolated workers.

Any updates on this subject ?

[jezzsantos]

Really struggling to find any examples that work for .NET 6, 7 or 8 using isolated workers. Now 18 months later. Come on, someone must know how to test these things properly?

[alexjamesbrown]

I noticed this appear in .net 8 and was hoping this was the answer https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.hosting.testing.fakehost?view=dotnet-plat-ext-8.0

Or:

https://github.com/dotnet/extensions/blob/f4315cd121a4b4006db2ba5744c602e4fef13508/src/Libraries/Microsoft.Extensions.Hosting.Testing/FakeHost.cs#L14

It's still in pre-release though.

I got some of the way there using the answer to this SO question

But now I get the error:

The gRPC channel URI 'http://:0' could not be parsed.

I've put my findings so far into a repo: https://github.com/alexjamesbrown/azure-functions-isolated-worker-integration-tests

[jezzsantos]

Hey @alexjamesbrown

You can get past the gRPC thing using this: services.RemoveAll<IHostedService>();

But then you will encounter the harder problem of function not being triggered (i.e QueueTriggers), which for me and others, is ultimately the real problem to solve.

This code, is as far as I got,

        _host = new HostBuilder()
            .ConfigureAppConfiguration(builder =>
            {
                builder
                    .AddJsonFile("appsettings.Testing.json", true);
            })
            .ConfigureAzureFunctionTesting<Program>()
            .ConfigureServices((_, services) =>
            {
                if (_overridenTestingDependencies.Exists())
                {
                    _overridenTestingDependencies.Invoke(services);
                }
            })
            .Build();
        _host.Start();

that uses this extension method that loads all the code that the AzureFunctions project loads on startup (created by the Source Generator):

internal static class AzureFunctionTestingExtensions
{
    /// <summary>
    ///     Configures the test process to load and run the azure functions
    /// </summary>
    public static IHostBuilder ConfigureAzureFunctionTesting<TProgram>(this IHostBuilder builder)
    {
        //HACK: this does not work yet, still waiting for the Azure Functions team to solve this problem:
        // https://github.com/Azure/azure-functions-dotnet-worker/issues/281
        return builder
            .ConfigureFunctionsWorkerDefaults()
            .InvokeAutoGeneratedConfigureMethods<TProgram>()
            .ConfigureServices((context, services) =>
            {
                services.RemoveAll<IHostedService>(); // We need remove this host to prevent gRPC running
                services.AddDependencies(context);
            });
    }

    /// <summary>
    ///     Invokes auto-generated configuration methods for a given <see cref="IHostBuilder" />.
    ///     This method searches for classes that implement the <see cref="IAutoConfigureStartup" /> interface in the assembly
    ///     of the specified type <see cref="TProgram" />.
    ///     This mimics what the <see cref="WorkerHostBuilderExtensions.ConfigureFunctionsWorkerDefaults(IHostBuilder)" />
    ///     method does on
    ///     startup in an Azure project
    /// </summary>
    private static IHostBuilder InvokeAutoGeneratedConfigureMethods<TProgram>(this IHostBuilder builder)
    {
        var startupTypes = typeof(TProgram).Assembly
            .GetTypes()
            .Where(t => typeof(IAutoConfigureStartup).IsAssignableFrom(t)
                        && t is { IsInterface: false, IsAbstract: false });

        foreach (var type in startupTypes)
        {
            var instance = (IAutoConfigureStartup)Activator.CreateInstance(type)!;
            instance.Configure(builder);
        }

        return builder;
    }
}

[Barsonax]

Would be nice if someone from Microsoft could step in and finally shed some light on how to do this.

We are even considering to move away from Azure functions for our APIs and use ASP .NET (which has webapplicationfactory) because without this we just cannot deliver the quality we want.

[steve-white-eo]

We are in the same boat. In our solution we typically have HttpTrigger functions in one project and a Client project which builds a Refit HttpClients package, that other services can consume.

We want to spin up the functions host to test the Refit clients against the actual function to ensure nothing is broken, end to end. This is trivial if we use an API.

[ssnseawolf]

.NET 9 hit GA, and with isolated functions moving to IHostApplicationBuilder I assumed simple WebApplicationFactory integration testing would be supported, which seemed like a logical assumption.

However, WebApplicationFactory doesn't pass in CLI arguments as mentioned earlier. Guidance, workarounds and perhaps a sprinkling of roadmap would be quite appreciated here. There's a lot of machinery at work and engineering a solution is challenging, so customers will see substantial benefit from official guidance or documentation.