Open maisiesadler opened 3 years ago
I think the problem here is that you're binding the configuration into your service while the application is still in its bootstrapping phase (which is until Build()
), so the configuration isn't yet "done". It's similar to the sort of issue where you need configuration to configure other things, like if you're say using Azure Key Vault, and need to use config to configure the Key Vault provider itself.
Does it work as you'd expect if you change it to this instead?
builder.Services.AddSingleton(() => new Options(builder.Configuration["AppSettings:Options"]));
I think if you resolve it lazily like that, when the lambda to get Options
runs the Configuration
would then have the extra settings in it from the tests.
Yeah it's probably more of a gotcha of the new model - the WebApplicationFactory
code gets invoked as part of a callback that gets called before Build()
returns back to the Program
class, so the code that adds your test settings won't even have run yet when you register that dependency so it can't see your overrides as they don't exist yet.
Makes sense. Will update the code and close the issue - thanks!
Reopening so we document this. This might also be worth trying to fix in .NET 7. Thanks for the great repro.
@halter73 are you gonna doc this?
Here's my question on SO on the same issue: https://stackoverflow.com/questions/69986598/with-webapplicationfactory-add-configuration-source-before-program-cs-executes
I stumbled upon a workaround for this issue.
If you override WebApplicationFactory<T>.CreateHost()
and call IHostBuilder.ConfigureHostConfiguration()
before calling base.CreateHost()
the configuration you add will be visible between WebApplication.CreateBuilder()
and builder.Build()
. From my testing it's visible in all the various places you'd look at the config (callbacks, etc.).
I have not tried it with adding a JSON file but I imagine it should work.
Note that calling IHostBuilder.ConfigureAppConfiguration()
in this manner does not work. DeferredHostBuilder
treats host configuration differently from app configuration. (Code part 1 - Code part 2)
The comments in DeferredHostBuilder.ConfigureHostConfiguration
actually say "We can do this for app configuration as well if it becomes necessary." Perhaps it's necessary now. 😄
(The PR that made ConfigureHostConfiguration()
work like this in DeferredHostBuilder
.)
var builder = WebApplication.CreateBuilder(args);
var result = builder.Configuration["Result"];
var app = builder.Build();
app.MapGet("/", () => result);
app.Run();
public partial class Program{ }
{
"Result" : "Normal Result"
}
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using NUnit.Framework;
namespace TestProject;
[TestFixture]
public class ProgramFixture
{
[Test]
public async Task Should_return_override()
{
var factory = new MyWebApplicationFactory();
var client = factory.CreateClient();
var result = await client.GetStringAsync("/");
Assert.That(result, Is.EqualTo("Override"));
}
private class MyWebApplicationFactory : WebApplicationFactory<Program>
{
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
{
config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
});
// This does not work.
//builder.ConfigureAppConfiguration(config =>
//{
// config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
//});
return base.CreateHost(builder);
}
// This does not work.
//protected override void ConfigureWebHost(IWebHostBuilder builder)
//{
// builder.ConfigureAppConfiguration(config =>
// {
// config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
// });
//}
}
}
The above workaround did not work for me because I wanted the AddInMemoryCollection
to override the existing config (ie: be added last). So here's what I came up with instead.
Program.cs:
var builder = WebApplication.CreateBuilder(args);
builder.WebHost.ConfigureAppConfiguration(builder =>
{
builder
.AddInMemoryCollection(s_ConfigOverride);
});
...
public partial class Program
{
static Dictionary<string, string> s_ConfigOverride { get; set; } = new();
class ClearConfigOverride : IDisposable
{
public void Dispose() => s_ConfigOverride = new Dictionary<string, string>();
}
public static IDisposable OverrideConfig(Dictionary<string, string> config)
{
s_ConfigOverride = config;
return new ClearConfigOverride();
}
}
UnitTest.cs
[OneTimeSetUp]
public void OneTimeSetup()
{
using var _ = Program.OverrideConfig(new Dictionary<string, string>()
{
{ "MyConfig", "someValue" }
});
var application = new WebApplicationFactory<Program>()
.WithWebHostBuilder(builder =>
{
// ... Configure test services
});
}
....
Having the same issue.
I'm able to set the environment to "Test" for my functional tests but when reading settings at runtime (during tests), the appsettings.Test.json
is being ignored when using minimal API
I use it like this:`
var application =
new WebApplicationFactory<Program>()
.WithWebHostBuilder(
config =>
config
.UseCommonConfiguration()
.UseEnvironment("Test")
.ConfigureTestServices(ConfigureTestServices));
And I have the extension
public static class WebHostBuilderExtensions
{
public static IWebHostBuilder UseCommonConfiguration(this IWebHostBuilder builder)
{
builder.ConfigureAppConfiguration((hostingContext, config) =>
{
var env = hostingContext.HostingEnvironment;
config
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: false)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: false)
.AddEnvironmentVariables();
if (env.IsDevelopment())
{
var appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
config.AddUserSecrets(appAssembly, optional: true);
}
});
return builder;
}
}
I don't understand why some of you suggest this is a gotcha rather than a bug.
If I set the webhost builder to use environment Test, I would expect the IConfiguration to read from the appsettings.Test.json
.
Also, why do you say it applies only after app has been built?
I can see my builder.Build()
happening before the IConfiguration
is retrieved from the DI container and settings read
This is my Program.cs
using Rubiko.Logging.Microsoft.DependencyInjection;
using Sample.Extensions;
const string corsPolicy = "corsPolicy";
var builder = WebApplication.CreateBuilder(args);
var configuration = builder.Configuration;
// IoCC
var services = builder.Services;
services
.AddMicrosoftFactoryLogging(configuration.GetSection("Logging"))
.AddMicroserviceDependencies()
.AddCors(corsPolicy);
var app = builder.Build();
// Http Pipeline
var assemblyPath = $"/{typeof(Program).Assembly.GetName().Name!.ToLowerInvariant()}";
app
.UsePathBase(assemblyPath)
.UseRouting()
.UseCors(corsPolicy)
.UseDeveloperExceptionPage()
.UseOpenApi(assemblyPath)
.UseAuthorization()
.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
app.Run();
// See https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-6.0
public partial class Program { }
If anyone has a workaround, please let me know. But I will probably go back to the old fashioned Startup.cs
and the
var server =
new TestServer(
new WebHostBuilder()
.UseStartup<Startup>()
.UseCommonConfiguration()
.UseEnvironment("Test")
.ConfigureTestServices(ConfigureTestServices));
This is blocking my effort to move to new ways of app init too.. Minimal code looks like a total anti-feature. To be able to show of a one-liner for hello world, we have to cope with a lot of new magic that complicates flexibility in unit testing, making it harder to make anything that is a real application..
Before .NET 6 we implement a main method, handle the command line arguments and call into web API init code. That way it is easy for us to make unit tests test everything and provide anything wanted from the unit test libraries into the application to mock, stub, and avoid running code that will blow up if not in production environment.
I'm unable to find any documentation of how to convert the minimal code to actual readable normal code which one can integrate with, and I have not been able to find a workaround to make a unit test to be able to provide input readable from the very beginning of the Program.cs file.
Going through a static variable of course works, but I want to avoid that as I want multiple unit tests to be able to run in parallel.
All I've managed to do so far is inject a service that can be fetched after builder.Build() has been called, but that is too late..
We're going to tackle this in .NET 8, it'll require some new APIs to make it work well.
Inside protected override void ConfigureWebHost(IWebHostBuilder builder)
var config = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string>() {})
.Build();
builder.UseConfiguration(config);
Appears to work as an alternative for the ConfigureAppConfiguration
but not sure if that Is desired.
Guidance on this would be appreciated especially if it would be delayed to .NET8 Thanks Max
Sad times.
Thanks for contacting us.
We're moving this issue to the .NET 8 Planning
milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.
I've faced with similar issue in my tests. I need to use class inherited from WebApplicationFactory<Program>
in my tests. The issue is that in Production scenario Program.cs
initializes Serilog with MSSqlServer sink. So I have to have to provide connectionString
- it isn't what I would like to do in tests. Part of my appsettings.json
(located in project under test) is:
"Serilog": { "Using": [ "Serilog.Sinks.MSSqlServer" ], "MinimumLevel": { ... } }, "WriteTo": [ { "Name": "MSSqlServer", "Args": { "connectionString": "{connection_string}", "sinkOptionsSection": { ... }, "columnOptionsSection": { "customColumns": [ ... ] } } } ] }
This answer prompted me to find workaround for my case:
internal class MyWebApplicationFactory : WebApplicationFactory<Program>
{
...
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
{
config.AddJsonFile("appsettings.json");
});
return base.CreateHost(builder);
}
...
}
where appsettings.json
(located in test project) is:
{ "Serilog": { "WriteTo": [ { "Name": "Console" }, { "Name": "Console" } ] } }
Set this advanced settings for your test project
So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?
So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?
I couldn't make it work either, so I abandoned minimal API and keep using the standard Startup + Program.
So I was able to get this to work with a kind-of-hacky workaround. My Program.cs looks something like this:
var app = WebApplication.CreateBuilder(args)
.BuildDefaultConfiguration()
.ConfigureServices((services, configuration) =>
{
//configure IServiceCollection
})
.Build()
.ConfigureApplication(app =>
{
app.UseHealthChecks("/hc", new HealthCheckOptions
{
Predicate = _ => true,
ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
});
//define your other routes and what not
});
await app.RunAsync();
and I have the following extension methods defined:
public static class BootstrappingExtensions
{
public static WebApplicationBuilder BuildDefaultConfiguration(this WebApplicationBuilder builder)
{
if(builder.Configuration.GetValue("IsConfigured", false)) return builder; // this is the hacky/magic part
//load appsettings.json or Azure App Config or whatever you want
return builder;
}
public static WebApplicationBuilder ConfigureServices(this WebApplicationBuilder builder, Action<IServiceCollection, IConfiguration> configure)
{
configure(builder.Services, builder.Configuration);
return builder;
}
public static WebApplication ConfigureApplication(this WebApplication app, Action<WebApplication> configure)
{
configure(app);
return app;
}
}
then in my test fixture I have:
[TestFixture]
public sealed class ProgramTest
{
private sealed class ProgramFactory : WebApplicationFactory<Program>
{
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration((configBuilder) =>
{
configBuilder.Sources.Clear();
configBuilder.AddInMemoryCollection(new Dictionary<string,string>
{
{ "ConnectionStrings:MyDatabaseConnection", "<some test value>" },
{ "IsConfigured", "true" }
});
});
}
}
[Test]
public async Task Program_HealthCheck_ShouldReturnHealthy()
{
using var application = new ProgramFactory();
using var client = application.CreateClient();
var response = await client.GetAsync("/hc");
var report = await response.Content.ReadAsJsonAsync<UIHealthReport>(); //ymmv as far as json options goes here
report.Should().NotBeNull().And.Match<UIHealthReport>(_ => _.Status == UIHealthStatus.Healthy);
}
}
So the downside obviously is you're setting a test flag and checking for it in production code, but it does seem to work for overriding the config with minimal APIs until a real fix comes along in .NET 8.
OK seeing the responses here urged me to come up with something in the meantime while we build a cleaner solution to this. The solution has 2 parts to it. A line of code you have to add to the application when you want to override configuration before:
Application
var builder = WebApplication.CreateBuilder(args);
// This is the special line of code. It should be added in the place where you want to override configuration
builder.Configuration.AddTestConfiguration();
var result = builder.Configuration["Result"];
var app = builder.Build();
app.MapGet("/", () => result);
app.Run();
public partial class Program { }
Test project
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;
namespace TestProject1
{
public class UnitTest1
{
[Fact]
public async Task OverrideWithInMemoryConfiguration()
{
var waf = new WebApplicationFactory<Program>();
TestConfiguration.Create(b => b.AddInMemoryCollection(new Dictionary<string, string?>
{
["Result"] = "Hello World!"
}));
var client = waf.CreateClient();
var response = await client.GetStringAsync("/");
Assert.Equal("Hello World!", response);
}
}
}
Here's the magic that makes this work:
namespace Microsoft.Extensions.Configuration;
internal static class TestConfiguration
{
// This async local is set in from tests and it flows to main
internal static readonly AsyncLocal<Action<IConfigurationBuilder>?> _current = new();
/// <summary>
/// Adds the current test configuration to the application in the "right" place
/// </summary>
/// <param name="configurationBuilder">The configuration builder</param>
/// <returns>The modified <see cref="IConfigurationBuilder"/></returns>
public static IConfigurationBuilder AddTestConfiguration(this IConfigurationBuilder configurationBuilder)
{
if (_current.Value is { } configure)
{
configure(configurationBuilder);
}
return configurationBuilder;
}
/// <summary>
/// Unit tests can use this to flow state to the main program and change configuration
/// </summary>
/// <param name="action"></param>
public static void Create(Action<IConfigurationBuilder> action)
{
_current.Value = action;
}
}
NOTE: This in the unit test, the call to Create must happen before the CreateClient() call on the WebApplicationFactory<T>
. This is because the ExecutionContext needs to flow to the main application and that gets initialized during the first access to the client or services on the application.
EDIT: If you derive a class from WebApplicationFactory
public class UnitTest1
{
[Fact]
public async Task ImplicitOverride()
{
var app = new App();
app.ConfigureConfiguration(b => b.AddInMemoryCollection(new Dictionary<string, string?>
{
["Result"] = "Hello!"
}));
var client = app.CreateClient();
var response = await client.GetStringAsync("/");
Assert.Equal("Hello!", response);
}
}
class App : WebApplicationFactory<Program>
{
private Action<IConfigurationBuilder>? _action;
public void ConfigureConfiguration(Action<IConfigurationBuilder> configure)
{
_action += configure;
}
protected override IWebHostBuilder? CreateWebHostBuilder()
{
if (_action is { } a)
{
// Set this so that the async context flows
TestConfiguration.Create(a);
}
return base.CreateWebHostBuilder();
}
}
This is a bit cleaner and lets you configure the WebApplicationFactory instead of a static.
Depending on the use-case, a less robust but fairly simple short term workaround could be to use Environment Variable configuration.
Environment.SetEnvironmentVariable("KEY", "VALUE");
var waf = new WebApplicationFactory<Program>();
var client = waf.CreateClient()
/* ... rest of test ... */
Edit: Don't do this, DavidFowl makes a really great point about how this could produce unexpected results.
Yea I don’t recommend that. It affects the entire process (including the test process). That’s no good, especially since tests run concurrently by default.
This seems to be the best workaround so far to override settings.
Also the .ConfigureServices
seems to be executed AFTER the Program.cs
registrations, so it's a good place to remove/replace service registrations.
With those two things it seems I can get by using mnimal API with .NET 7 without sacrificing integration tests or messing around with environment variables
Example:
protected IntegrationTestBase()
{
// Workaround for overrides in .NET 7 and minimal API. See https://github.com/dotnet/aspnetcore/issues/37680
var testConfigurationBuilder =
new ConfigurationBuilder()
.AddInMemoryCollection(
new List<KeyValuePair<string, string?>>
{
new("CoolSection:Colors:0", "red"),
new("CoolSection:Colors:1", "black"),
})
.Build();
var webApplicationFactory =
new WebApplicationFactory<Program>() // It requires public partial class Program { } at Program.cs in main assembly
.WithWebHostBuilder(webHostBuilder =>
webHostBuilder
.UseEnvironment("Development")
.UseConfiguration(testConfigurationBuilder)
.ConfigureServices(TestServiceOverrides));
// ...
}
protected virtual void TestServiceOverrides(IServiceCollection servives)
{
// Replace services in DI container. Virtual method so that test classes inheriting from IntegrationTestBase can override services also
}
Let's hope .NET 8 brings some good IWebHostedBuilder
methods that work for these purposes. Because ConfigureTestServices
and ConfigureAppConfiguration
right now are misleading.
I just ran into this issue and wanted to add my 2 cents. It seems as if the behavior isn't consistent across different operating systems?
builder.ConfigureAppConfiguration
worked for me in Windowsbuilder.ConfigureHostConfiguration
for it to work in WSL (Ubuntu) - this also still worked in Windowsbuilder.ConfigureHostConfiguration
also worked when running tests in the mcr.microsoft.com/dotnet/sdk:7.0
Docker image but it wasn't loading the environment specific appsettings
JSON config so the exceptions being thrown in WSL and Docker were differentAnyone stumbling upon this issue, just rewrite your application to use Program.cs
&Startup.cs
, close this tab and be productive again, its that simple.
I’ll assign this one to myself
I have just discovered why any of the above was NOT working in my solution.
I was creating builder in my "Program.cs" like this:
var builder = WebApplication.CreateBuilder();
I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.
Then I modified my Program.cs with this:
var builder = WebApplication.CreateBuilder(args);
And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)
I have just discovered why any of the above was NOT working in my solution.
I was creating builder in my "Program.cs" like this:
var builder = WebApplication.CreateBuilder();
I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.
Then I modified my Program.cs with this:
var builder = WebApplication.CreateBuilder(args);
And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)
omg thank you so much!
i was losing my mind trying to hunt down why the environment set in WAF wasn't being picked up by the application.
all i had to do was add the args
to WebApplication.CreateBuilder
method.
so anyone reading this on .NET 7+ just make sure you create your app builder like this:
var builder = WebApplication.CreateBuilder(args);
Some ppl seem to imply/hope this is getting fixed in .NET 8. Can anyone confirm this is actually getting fixed in .NET 8? This is not just a gotcha but a breaking change in behavior between the old startup and the new WebApplication way of doing things. All the workarounds posted here seem to be pretty hacky and not a true solution to the problem.
It was not fixed in .NET 8.
I have just discovered why any of the above was NOT working in my solution.
I was creating builder in my "Program.cs" like this:
var builder = WebApplication.CreateBuilder();
I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.
Then I modified my Program.cs with this:
var builder = WebApplication.CreateBuilder(args);
And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)
OMG! 🤯 I spent nearly 2 days going through every possible permutation without success. This actually makes the WAF config work on dotnet 6! I had already resigned to using David's temporary workaround until I tried this.
Here's my case: I couldn't get AddUserSecrets to load anything, which was problematic since it contains our AWS configuration settings.
public class TestBase : WebApplicationFactory<Program>
{
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(configBuilder =>
{
configBuilder.AddJsonFile("appsettings.json", false);
configBuilder.AddUserSecrets<Program>(true, true);
});
builder.ConfigureServices(ConfigureServices);
return base.CreateHost(builder);
}
protected virtual void ConfigureServices(IServiceCollection services)
{
}
}
I want to add my experience in case anyone finds it useful:
i'm using .net 8 with testcontainers an therefore need to add my configurations, i thought i should clear the sources before adding news ones but i was wrong, just removing the line fixed the issue for my use case.
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureHostConfiguration(config =>
{
//config.Sources.Clear(); <-- removed this line right here
var configuration = new ConfigurationBuilder()
.AddUserSecrets(typeof(CustomWebApplicationFactory<TProgram>).Assembly, true, true)
.Build();
config.AddConfiguration(configuration);
config.Sources.Add(new MemoryConfigurationSource
{
InitialData = new Dictionary<string, string>
{
{"ConnectionStrings:Database", _database.GetConnectionString().Replace("Database", "Initial Catalog")},
{"ConnectionStrings:Storage", _storage.GetConnectionString()},
{"ConnectionStrings:Redis:Host", _redis.Hostname},
{"ConnectionStrings:Redis:Password", string.Empty}
}!
});
});
return base.CreateHost(builder);
}
@davidfowl This was not mentioned in any Asp.Net Core 9 "what we are doing" posts. I'm assuming that's mostly because this is a small change/bug fix? This is still in the plans, right?
I'm asking since it's still in the .Net 8 Planning
Milestone, despite it being launched months ago, and this doesn't feel like a "Fix in Servicing" kind of issue.
Is it still planned for dotnet 9? It has net9 planning milestone, but not included in any net9 roadmaps. Thanks!
I'm fairly sure at this point it will have missed the boat for inclusion in .NET 9 given no one appears to have done any work on it at all.
This seems to be the best workaround so far to override settings.
Also the
.ConfigureServices
seems to be executed AFTER theProgram.cs
registrations, so it's a good place to remove/replace service registrations.With those two things it seems I can get by using mnimal API with .NET 7 without sacrificing integration tests or messing around with environment variables
Example:
protected IntegrationTestBase() { // Workaround for overrides in .NET 7 and minimal API. See https://github.com/dotnet/aspnetcore/issues/37680 var testConfigurationBuilder = new ConfigurationBuilder() .AddInMemoryCollection( new List<KeyValuePair<string, string?>> { new("CoolSection:Colors:0", "red"), new("CoolSection:Colors:1", "black"), }) .Build(); var webApplicationFactory = new WebApplicationFactory<Program>() // It requires public partial class Program { } at Program.cs in main assembly .WithWebHostBuilder(webHostBuilder => webHostBuilder .UseEnvironment("Development") .UseConfiguration(testConfigurationBuilder) .ConfigureServices(TestServiceOverrides)); // ... } protected virtual void TestServiceOverrides(IServiceCollection servives) { // Replace services in DI container. Virtual method so that test classes inheriting from IntegrationTestBase can override services also }
Let's hope .NET 8 brings some good
IWebHostedBuilder
methods that work for these purposes. BecauseConfigureTestServices
andConfigureAppConfiguration
right now are misleading.
Just dropping into say this worked for me.
Describe the bug
Using .net 6 minimal APIs and WebApplicationFactory for testing, the test config override is only applied after .Build. This means any interaction with config while configuring services using
builder.Configuration
will be using the app configuration, not test overrides.To Reproduce
I've created a repo that shows the difference, here is the Program, and an Example test.
Test run
Further technical details