multiple JwtBearer authentication schemes continually refresh metadata

acblksun commented 5 years ago

Describe the bug

We require accepting multiple JwtBearer audience/authority pairs for our api.

When we setup multiple JwtBearer authentication for our api, we notice that aspnetcore middleware is continually calling the .well-known/openid-configuration at the minimum refresh interval of 30 seconds instead of the desired AutomaticRefreshInterval of 1 day.

To Reproduce

Steps to reproduce the behavior:

Using ASP.NET Core Version 2.2
Configure to use multiple Jwt Bearers (see code below)
Get a valid authorization token (note - invalid tokens have the same result)
Run any endpoint that engages the middleware e.g. [Authorize] and pass the token correctly
Observe that the .well-known/openid-configuration for all configured JwtBearer's will be called every 30 seconds (or every RefreshInterval) instead of once a day (or every AutomaticRefreshInterval).

Expected behavior

Expect the default Refresh Interval of 30 seconds, and AutomaticRefreshInterval of 1 day to be sufficient, and the .well-known/openid-configuration to be called at most once per day per scheme.

Note that I did perform a test of a single Jwt Bearer registered as the default, and it did perform as expected only calling at the AutomaticRefreshInterval

Additional context

using the http interceptor isn't required if you can monitor with Application Insights
it makes no difference if the audience strings are the same or different

Include the output of dotnet --info

C:\repos\test>dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.401
 Commit:    729b316c13

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.2.401\

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  1.0.2 [C:\Program Files\dotnet\sdk]
  1.0.3 [C:\Program Files\dotnet\sdk]
  1.0.4 [C:\Program Files\dotnet\sdk]
  1.1.0 [C:\Program Files\dotnet\sdk]
  2.0.2 [C:\Program Files\dotnet\sdk]
  2.0.3 [C:\Program Files\dotnet\sdk]
  2.1.2 [C:\Program Files\dotnet\sdk]
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.101 [C:\Program Files\dotnet\sdk]
  2.1.102 [C:\Program Files\dotnet\sdk]
  2.1.103 [C:\Program Files\dotnet\sdk]
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.200 [C:\Program Files\dotnet\sdk]
  2.1.201 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.302 [C:\Program Files\dotnet\sdk]
  2.1.400 [C:\Program Files\dotnet\sdk]
  2.1.401 [C:\Program Files\dotnet\sdk]
  2.1.402 [C:\Program Files\dotnet\sdk]
  2.1.403 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]
  2.1.502 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.1.504 [C:\Program Files\dotnet\sdk]
  2.1.505 [C:\Program Files\dotnet\sdk]
  2.1.507 [C:\Program Files\dotnet\sdk]
  2.1.700 [C:\Program Files\dotnet\sdk]
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.102 [C:\Program Files\dotnet\sdk]
  2.2.300 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.3-servicing-26724-03 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

Code to reproduce. I used the VS2019 template for a webapi and modified it slightly.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.HttpsPolicy;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace WebApplicationAuthTest
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            // this example uses 2 different auth0 tenants, but this shouldn't matter
            const string audience_one = "https://api1.example.com";
            const string authority_one = "https://test-tenant-a.au.auth0.com/";
            const string audience_two = "https://api2.example.com";
            const string authority_two = "https://test-tenant-b.au.auth0.com/";

            var auth = services
                .AddAuthentication()
#if true
                .AddJwtBearerWithHttpIntercept("Auth-A", audience_one, authority_one)
                .AddJwtBearerWithHttpIntercept("Auth-B", audience_two, authority_two)
#else
                // AddJwtBearerWithHttpIntercept is equivalent to these. All it adds is http client interception for request/response logging.
                .AddJwtBearer("Auth-A", options =>
                {
                    options.Audience = audience_one;
                    options.Authority = authority_one;

                    options.Validate();
                })
                .AddJwtBearer("Auth-B", options =>
                {
                    options.Audience = audience_two;
                    options.Authority = authority_two;

                    options.Validate();
                })
#endif
                ;

            services.AddAuthorization(options =>
            {
                options.DefaultPolicy = new AuthorizationPolicyBuilder()
                    .RequireAuthenticatedUser()
                    .AddAuthenticationSchemes("Auth-A", "Auth-B")
                    .Build();
            });

            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseMvc();
        }
    }

    public static class JwtConfiguration
    {
        public static AuthenticationBuilder AddJwtBearerWithHttpIntercept(this AuthenticationBuilder authenticationBuilder, string schemeIdentifier, string audience, string authority)
        {
            return authenticationBuilder
                 .AddJwtBearer(schemeIdentifier, options =>
                 {
                     options.Audience = audience;
                     options.Authority = authority;

                     // Except for this - we added it ourselves. Technically all we should need to set is this unless we want to modify configurationManager.
                     options.BackchannelHttpHandler = new LoggingHandler(new System.Net.Http.HttpClientHandler());

#if false

                     // this is as-is defaults from https://github.com/aspnet/AspNetCore/blob/v2.2.6/src/Security/Authentication/JwtBearer/src/JwtBearerPostConfigureOptions.cs
                     if (string.IsNullOrEmpty(options.MetadataAddress) && !string.IsNullOrEmpty(options.Authority))
                     {
                         options.MetadataAddress = options.Authority;
                         if (!options.MetadataAddress.EndsWith("/", StringComparison.Ordinal))
                         {
                             options.MetadataAddress += "/";
                         }

                         options.MetadataAddress += ".well-known/openid-configuration";
                     }

                     var httpClient = new System.Net.Http.HttpClient(options.BackchannelHttpHandler ?? new System.Net.Http.HttpClientHandler());
                     httpClient.Timeout = options.BackchannelTimeout;
                     httpClient.MaxResponseContentBufferSize = 1024 * 1024 * 10; // 10 MB

                     var configurationManager = new Microsoft.IdentityModel.Protocols.ConfigurationManager<Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfiguration>(
                         options.MetadataAddress, new Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever(),
                         new Microsoft.IdentityModel.Protocols.HttpDocumentRetriever(httpClient) { RequireHttps = options.RequireHttpsMetadata });
                     options.ConfigurationManager = configurationManager;

                     // Except here - we manually increase the RefreshInterval to prove it's the RefreshInterval
                     configurationManager.RefreshInterval = TimeSpan.FromMinutes(5);
                     // configurationManager.AutomaticRefreshInterval = 
#endif

                     options.Validate();
                     Console.WriteLine($"Added {schemeIdentifier} for authority {authority} and audience {audience}");

                 });
        }
    }

    public class LoggingHandler : System.Net.Http.DelegatingHandler
    {
        public LoggingHandler(System.Net.Http.HttpMessageHandler innerHandler)
            : base(innerHandler)
        {
        }

        protected override async Task<System.Net.Http.HttpResponseMessage> SendAsync(System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
        {
            Console.WriteLine("Request:");
            Console.WriteLine(request.ToString());
            if (request.Content != null)
            {
                Console.WriteLine(await request.Content.ReadAsStringAsync());
            }
            Console.WriteLine();

            System.Net.Http.HttpResponseMessage response = await base.SendAsync(request, cancellationToken);

            Console.WriteLine("Response:");
            Console.WriteLine(response.ToString());
            if (response.Content != null)
            {
                Console.WriteLine(await response.Content.ReadAsStringAsync());
            }
            Console.WriteLine();

            return response;
        }
    }
}

Tratcher commented 5 years ago

Does it hit this code path? https://github.com/aspnet/AspNetCore/blob/71c5c66b211c7ab9d44751e39cef27c525124bb7/src/Security/Authentication/JwtBearer/src/JwtBearerHandler.cs#L112-L121 A) Token arrives for Auth-B. B) Authenticate runs for Auth-A, fails with SecurityTokenSignatureKeyNotFoundException (?), requests a rate limited refresh of metadata C) Authenticate runs for Auth-B and succeeds.

If this is what's happening then you should only see refreshes for A, not B. Disabling RefreshOnIssuerKeyNotFound would prevent it.

HudsonAkridge commented 5 years ago

We're having the exact same problem where RefreshOnIssuerKeyNotFound is firing on either Bearer Scheme failing. One of the two will always fail for us, while the other succeeds.

Tratcher commented 5 years ago

@HudsonAkridge it should only ever fire for the first provider. Can you confirm?

eliaslopezgt commented 5 years ago

@Tratcher If I add 4 JWTBearerHandlers and only one can validate the token, it reloads the configuration on the other 3. Hope that helps. https://github.com/aspnet/AspNetCore/issues/14397

blowdart commented 5 years ago

Does it happen every time? Every so often isn't unsurprising as it will go through the handlers in order and refresh to see if there's a new key.

eliaslopezgt commented 5 years ago

Maybe this section from the issue I created helps.

Expected behavior

This behavior is acceptable as long as there's only one JWTBearerHandler. Having multiple JWTBearerHandlers results in unwanted traffic to the configuration endpoints. This can be currently mitigated by:

Setting the RefreshInterval to something different than its default (30 seconds)
Setting RefreshOnIssuerKeyNotFound to false in all the JWTBearerHandlers

My proposed solution is to create an additional OnSignatureValidationFailed event that can be triggered before Options.ConfigurationManager.RequestRefresh(); in JwtBearerHandler.cs https://github.com/aspnet/AspNetCore/blob/master/src/Security/Authentication/JwtBearer/src/JwtBearerHandler.cs

This with the purpose of allowing the caller to intercept the RequestRefresh and in this case, inject logic that can compare the invalid token audience and authority, with the audience and authority of the handler, if they are the same, then trigger the refresh otherwise it means we are trying to validate a token that is going to fail the validation no matter if we refresh.

The only change to the JWTBearerHandler is the addition of the OnSignatureValidationFailed event. I have a PR ready but wanted to discuss if this could bring any value or not.

Tratcher commented 5 years ago

@blowdart this is functioning as intended, and the mitigations listed above are effective:

Setting the RefreshInterval to something different than its default (30 seconds)
Setting RefreshOnIssuerKeyNotFound to false in all the JWTBearerHandlers

One way to redesign this would be to build one JwtAuthHandler that could contain multiple configurations. It would loop through all of them before triggering this kind of failure. Similar logic already exists when there are multiple token validators configured.

https://github.com/aspnet/AspNetCore/blob/752d99ca531f587fae92da63c1120f95c453e72a/src/Security/Authentication/JwtBearer/src/JwtBearerHandler.cs#L103-L105

Recommend backlog.

eliaslopezgt commented 5 years ago

@blowdart can I help with it?

Tratcher commented 5 years ago

@eliaslopezgt feel free to make some proposals, but for a refactor this large we want to agree on a design before starting any PRs.

Tiberriver256 commented 2 years ago

If anyone else is struggling with this, I stumbled across a workaround for this issue here: https://oliviervaillancourt.com/posts/Fixing-IDX10501-MultipleAuthScheme

Basically, the issue seems to be that when running multiple auth schemes every auth scheme will attempt to validate the token. For the correct auth scheme everything happens as expected. For the other auth schemes, signature validation will fail with SecurityTokenSignatureKeyNotFoundException: IDX10501: Signature validation failed. Unable to match key since the jwt token was issued by a different Idp. This marks the auth scheme with the RefreshOnIssuerKeyNotFound and the metadata is forced to reload.

The solution from the blog is basically to only attempt token validation if the issuer on the token matches the authority issuer:

var token = jwtHandler.ReadJwtToken(jwtToken);
if (string.Equals(token.Issuer, authorityIssuer, StringComparison.OrdinalIgnoreCase))
{
    // means the token was issued by this authority, we make sure full validation runs as normal
    return await base.HandleAuthenticateAsync();
}
else
{
    // Skip validation since the token as issued by a an issuer that this instance doesn't know about
    // That has zero of success, so we will not issue a "fail" since it crowds the logs with failures of type IDX10501 
    // which are not really true and certainly not useful.
    this.Logger.LogDebug($"Skipping jwt token validation because token issuer was {token.Issuer} but the authority issuer is: {authorityIssuer}");
    return AuthenticateResult.NoResult();
}

As a side note, it also gets rid of the problem in #18940

dotnet / aspnetcore