Options not resolved properly from WebApplicationFactory configuration

[span]

Describe the bug

When adding configuration files through the WebApplicationFactory the options for token acqusition are not set correctly. A null reference exception is thrown when trying to resolve the correct options.

When running the application host and using the health check to make an authenticated call, the options are resolved as expected and a token is acquired.

When running the test application, the exception is thrown.

Using Microsoft.Identity.Web 1.10 the tests work fine, but the changes introduced in 1.11 breaks the test integration.

To Reproduce

See repo for full example: https://github.com/span/aspnetcoreoptions

Short version:

Repro

Startup (With AzureAd config block in appsettings)

            services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
                .AddMicrosoftIdentityWebApi(Configuration)
                .EnableTokenAcquisitionToCallDownstreamApi()
                .AddInMemoryTokenCaches();

Message handler (used to fetch access token for a certain scope)

        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            var accessToken = await tokenAcquisition.GetAccessTokenForAppAsync(this.scope, authenticationScheme: "Bearer");

Expected behavior Access token received from IdP configured in AzureAd.

Actual behavior Null reference exception with stack trace:

System.NullReferenceException
Object reference not set to an instance of an object.
   at Microsoft.Identity.Web.MergedOptions.PrepareAuthorityInstanceForMsal()
   at Microsoft.Identity.Web.TokenAcquisition.BuildConfidentialClientApplication(MergedOptions mergedOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetOrBuildConfidentialClientApplication(MergedOptions mergedOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetAuthenticationResultForAppAsync(String scope, String authenticationScheme, String tenant, TokenAcquisitionOptions tokenAcquisitionOptions)
   at Microsoft.Identity.Web.TokenAcquisition.GetAccessTokenForAppAsync(String scope, String authenticationScheme, String tenant, TokenAcquisitionOptions tokenAcquisitionOptions)
   at Promotions.Host.HealthChecks.BearerHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /Users/daniel/mio/repos/Mio.Services/Mio.Promotions.API/src/Promotions.Host/HealthChecks/BearerHttpMessageHandler.cs:line 28
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)

### Further technical details
.NET SDK (reflecting any global.json):
 Version:   5.0.103
 Commit:    72dec52dbd

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.0
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /usr/local/share/dotnet/sdk/5.0.103/

Host (useful for support):
  Version: 5.0.3
  Commit:  c636bbdc8a

.NET SDKs installed:
  3.1.403 [/usr/local/share/dotnet/sdk]
  5.0.101 [/usr/local/share/dotnet/sdk]
  5.0.103 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.3 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

[span]

After further debugging I find that options are resolved properly when reading directly from IConfiguration in the message handler. The ITokenAcquisition still gets null values in its options monitor though.

Perhaps this should be redirected to Microsoft.Identity.Web?

[adityamandaleeka]

Can you please share the output of (IConfigurationRoot)Configuration.GetDebugView()? You'll need to run that in the app and then launch the test to see what the configuration is when the test runs.

[span]

Can you please share the output of (IConfigurationRoot)Configuration.GetDebugView()? You'll need to run that in the app and then launch the test to see what the configuration is when the test runs.

This is the output with some stuff redacted. The configuration looks fine as all the AzureAd entries are set as expected.

  CFBundleIdentifier=com.jetbrains.rider-EAP (EnvironmentVariablesConfigurationProvider)
  CF_USER_TEXT_ENCODING=0x1F5:0x0:0x2 (EnvironmentVariablesConfigurationProvider)
ANDROID_HOME=/usr/local/share/android-sdk (EnvironmentVariablesConfigurationProvider)
applicationName=WebApplication (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
AzureAd:
  ClientId=<REDACTED> (JsonConfigurationProvider for 'testsettings.json' (Required))
  ClientSecret=<REDACTED> (JsonConfigurationProvider for 'testsettings.json' (Required))
  Instance=https://login.microsoftonline.com/ (JsonConfigurationProvider for 'testsettings.json' (Required))
  TenantId=<REDACTED> (JsonConfigurationProvider for 'testsettings.json' (Required))
CLI_TELEMETRY_OPTOUT=1 (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
COMMAND_MODE=unix2003 (EnvironmentVariablesConfigurationProvider)
contentRoot=/Users/daniel/RiderProjects/aspnetcoretestoptions/WebApplication (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
DOTNET_CLI_TELEMETRY_OPTOUT=1 (EnvironmentVariablesConfigurationProvider)
DOTNET_ROOT=/usr/local/share/dotnet (EnvironmentVariablesConfigurationProvider)
DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 (EnvironmentVariablesConfigurationProvider)
environment=Development (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
HOME=/Users/daniel (EnvironmentVariablesConfigurationProvider)
JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home (EnvironmentVariablesConfigurationProvider)
LC_CTYPE=en_GB.UTF-8 (EnvironmentVariablesConfigurationProvider)
LESS=-R (EnvironmentVariablesConfigurationProvider)
Logging:
  LogLevel:
    Default=Information (JsonConfigurationProvider for 'appsettings.Development.json' (Optional))
    Microsoft=Warning (JsonConfigurationProvider for 'appsettings.Development.json' (Optional))
    Microsoft.Hosting.Lifetime=Information (JsonConfigurationProvider for 'appsettings.Development.json' (Optional))
LOGNAME=daniel (EnvironmentVariablesConfigurationProvider)
LSCOLORS=Gxfxcxdxbxegedabagacad (EnvironmentVariablesConfigurationProvider)
NVM_BIN=/Users/daniel/.nvm/versions/node/v10.23.0/bin (EnvironmentVariablesConfigurationProvider)
NVM_CD_FLAGS=-q (EnvironmentVariablesConfigurationProvider)
NVM_DIR=/Users/daniel/.nvm (EnvironmentVariablesConfigurationProvider)
NVM_INC=/Users/daniel/.nvm/versions/node/v10.23.0/include/node (EnvironmentVariablesConfigurationProvider)
OLDPWD=/ (EnvironmentVariablesConfigurationProvider)
P9K_SSH=0 (EnvironmentVariablesConfigurationProvider)
PAGER=less (EnvironmentVariablesConfigurationProvider)
PATH=<REDACTED>
PWD=/Users/daniel/RiderProjects/aspnetcoretestoptions/TestProject1/bin/Debug/net5.0 (EnvironmentVariablesConfigurationProvider)
ROOT=/usr/local/share/dotnet (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
Scope=<REDACTED> (JsonConfigurationProvider for 'testsettings.json' (Required))
SHELL=/bin/zsh (EnvironmentVariablesConfigurationProvider)
SKIP_FIRST_TIME_EXPERIENCE=1 (Microsoft.Extensions.Configuration.ChainedConfigurationProvider)
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.826vdPHXo8/Listeners (EnvironmentVariablesConfigurationProvider)
TERM=xterm-256color (EnvironmentVariablesConfigurationProvider)
TMPDIR=/var/folders/6s/_gbfjh1157j8_hnsqbj3z_rr0000gn/T/ (EnvironmentVariablesConfigurationProvider)
USER=daniel (EnvironmentVariablesConfigurationProvider)
XPC_FLAGS=0x0 (EnvironmentVariablesConfigurationProvider)
XPC_SERVICE_NAME=application.com.jetbrains.rider-EAP.10876746.10881686 (EnvironmentVariablesConfigurationProvider)
ZSH=/Users/daniel/.oh-my-zsh (EnvironmentVariablesConfigurationProvider)
_NO_DEBUG_HEAP=1 (EnvironmentVariablesConfigurationProvider)

Perhaps the way I am creating the scope, client and handler somehow bypasses the call that sets the options into the ITokenAcquisition. I noticed that when I run the application, I can get a breakpoint in the configuration where the optionsmonitor is updated and created. Looking at the call trace, it is called by the authentication middleware. The configuration setup is located inMicrosoftIdentityWebApiAuthenticationBuilderExtensions.AddMicrosoftIdentityWebApiImplementation lines 180 and forward.

When I run through the test fixture, the breakpoint never gets hit since the pipeline never receives a request and thus do not get processed. Perhaps that means that DI never loads the options into the monitor.

[span]

As a workaround for now I was able to "create" the options by calling Get on the IOptionsMonitor<JwtBearerOptions> for my scheme "Bearer". When calling Get, the configure actions is called and the options are set up correctly. This means an instance of the options will be available from the options monitor cache to TokenAcqusition when DI instantiates it.

A bit bulky but works. It is still not clear to me if this is a bug, feature or bad testing strategy.

            // Arrange
            using var scope = _factory.Server.Services.CreateScope();

            var monitor = scope.ServiceProvider.GetRequiredService<IOptionsMonitor<JwtBearerOptions>>();
            monitor.Get("Bearer");

            var handler = scope.ServiceProvider.GetRequiredService<BearerHttpMessageHandler>();
            var client = _factory.CreateDefaultClient(handler);

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.