tsmeets2
commented
2 years ago I suspect this is the same problem
https://github.com/AzureAD/microsoft-identity-web/discussions/1797
Open sipsorcery opened 2 years ago
tsmeets2
commented
2 years ago I suspect this is the same problem
https://github.com/AzureAD/microsoft-identity-web/discussions/1797
jmprieur
commented
2 years ago You need to use multiple authentication schemes. See https://github.com/AzureAD/microsoft-identity-web/wiki/multiple-authentication-schemes to specify a different auth scheme for Id.Web and the JwtBearer handler you have (for the moment this will be the same, that is JwtBearerDefaults.AuthenticationScheme)
nit: MSAL does not validate any token. It's a token acquisition library. Identity.Model does validate the tokens directed by the JwtBearer handler (part of ASP.NET Core), or Microsoft.Identity.Web (leveraging the JwtBearer handler)
sipsorcery
commented
2 years ago Thanks for the response.
I did spot the multiple authentication schemes artcile and attempted all the permutations I could come up with but in each case the Azure AD JWT token authentication was rendered inoperable.
For example swtiching from the block shown above to the one below results in still being able to authenticate Auth0, and my custom JWT tokens, but not Azure AD JWT tokens.
services.AddAuthentication().AddJwtBearer(AUTH0_AUTHENTICATION_SCHEME, options =>
{
options.Authority = domain;
options.TokenValidationParameters = new TokenValidationParameters
{
NameClaimType = ClaimTypes.NameIdentifier,
ValidateIssuer = true,
ValidateIssuerSigningKey = true,
ValidAudiences = new List<string>
{
Configuration[ConfigKeys.AUTH0_AUDIENCE],
Configuration[ConfigKeys.AUTH0_AUDIENCE_STRONG],
Configuration[ConfigKeys.AUTH0_AUDIENCE_MACHINE]
}
};
})
.AddJwtBearer("myapp", options =>
{
var key = Encoding.UTF8.GetBytes("secret-token-encryption-key");
options.TokenValidationParameters = new TokenValidationParameters
{
RequireSignedTokens = true,
ValidateIssuerSigningKey = true,
ValidateLifetime = true,
ValidateAudience = false,
ValidateIssuer = false,
RequireExpirationTime = false,
IssuerSigningKey = new SymmetricSecurityKey(key),
};
});
services.AddAuthentication()
.AddMicrosoftIdentityWebApi(Configuration.GetSection("AzureAd"), Microsoft.Identity.Web.Constants.AzureAd);As an additional experiement I did hack the code from AddMicrosoftIdentityWebApiImplementation into Startup.cs and was able to reduce the volume of token vaidation error messages.
services.AddHttpContextAccessor();
services.AddHttpClient();
services.AddSingleton<MicrosoftIdentityIssuerValidatorFactory>();
services.AddRequiredScopeAuthorization();
services.AddRequiredScopeOrAppPermissionAuthorization();
services.AddOptions<AadIssuerValidatorOptions>();
var authBuilder = services.AddAuthentication();
services.AddAuthentication().AddJwtBearer(AUTH0_AUTHENTICATION_SCHEME, options =>
{
options.Authority = domain;
options.TokenValidationParameters = new TokenValidationParameters
{
NameClaimType = ClaimTypes.NameIdentifier,
ValidateIssuer = true,
ValidateIssuerSigningKey = true,
ValidAudiences = new List<string>
{
Configuration[ConfigKeys.AUTH0_AUDIENCE],
Configuration[ConfigKeys.AUTH0_AUDIENCE_STRONG],
Configuration[ConfigKeys.AUTH0_AUDIENCE_MACHINE]
}
};
})
.AddJwtBearer("myapp", options =>
{
var key = Encoding.UTF8.GetBytes("secret-token-encryption-key");
options.TokenValidationParameters = new TokenValidationParameters
{
RequireSignedTokens = true,
ValidateIssuerSigningKey = true,
ValidateLifetime = true,
ValidateAudience = false,
ValidateIssuer = false,
RequireExpirationTime = false,
IssuerSigningKey = new SymmetricSecurityKey(key),
};
});
authBuilder.Services.AddOptions<JwtBearerOptions>(JwtBearerDefaults.AuthenticationScheme)
.Configure<IServiceProvider, IOptionsMonitor<MicrosoftIdentityOptions>, IOptions<MicrosoftIdentityOptions>>((
options,
serviceProvider,
msIdOptionsMonitor,
msIdOptions) =>
{
MicrosoftIdentityIssuerValidatorFactory microsoftIdentityIssuerValidatorFactory =
serviceProvider.GetRequiredService<MicrosoftIdentityIssuerValidatorFactory>();
options.Authority = "https://login.microsoftonline.com/CC6AEF0E-C60A-4BE0-80DE-F4499420F159/v2.0";
options.TokenValidationParameters = new TokenValidationParameters
{
TryAllIssuerSigningKeys = false,
IssuerValidator = microsoftIdentityIssuerValidatorFactory.GetAadIssuerValidator(options.Authority).Validate
};
});
services.AddMicrosoftIdentityWebApiAuthentication(Configuration, Microsoft.Identity.Web.Constants.AzureAd);Failed token log messages reduced to:
fail: Microsoft.IdentityModel.LoggingExtensions.IdentityLoggerAdapter[0]
IDX10500: Signature validation failed. No security keys were provided to validate the signature.
warn: Microsoft.IdentityModel.LoggingExtensions.IdentityLoggerAdapter[0]
IDX10233: ValidateAudience property on ValidationParameters is set to false. Exiting without validating the audience.I suspect this is the same problem
1797
Yep looks to be exactly the same problem.
sipsorcery
commented
2 years ago As @tsmeets2 states in his discussion post this issue was introduced in version 1.25.1. Version 1.25.0 works as expected.
tsmeets2
commented
2 years ago As @tsmeets2 states in his discussion post this issue was introduced in version
1.25.1. Version1.25.0works as expected.
The quick solution for me was to ignore those messages. But for me that is ugly.
jmprieur
commented
2 years ago @sruke, FYI, now that Ms.Id.Web integrated the identity logger there seem to be unwanted messages logged for token validation
nlz242
commented
2 years ago This is also problematic for me. We have an azure function app for which the users are authenticated thru 2 different azure ad, but also has a two downstream apis which are authenticated thru app registrations. Tokens are validated against all 4 schemes, all the time so it pollutes the logs like crazy. While this isn't great, it is even worse because those logs end up in our App Insight and they generate additionnal costs. We will be downgrading to 1.25 while this is being investigated but it is an important issue to us.
jmprieur
commented
2 years ago @sruke : do you think that the version of IdentityModel that released yesterday fix the issue?
sruke
commented
2 years ago @jmprieur, 6.22.0 version you're referring to will display log arguments like issuer, audience in clear text to display more meaningful messages.
The issue here seems to be different. It looks like Microsoft.IdentityModel is attempting to validate tokens that aren't from AzureAD which is causing the unwanted messages. Is it possible to instruct at Id.Web level to ignore tokens that aren't issued by AzureAD? Can multiple auth schemes solve this problem and is it being used correctly in the sample mentioned by sipsorcery?
jmprieur
commented
2 years ago @sipsorcery, @tsmeets2 @nlz242
Did you consider changing the log level? See: https://github.com/AzureAD/microsoft-identity-web/wiki/Logging
Thanks @sruke for looking into this.
nlz242
commented
2 years ago @jmprieur Seeing as the token validation, in the case where we use many Schemes, seems to be on a "try until it works, or all configured schemes have been tried" basis, the fact that these logs surface produces alot of logs for things that are not relevant.
For example, my App supports two different Azure B2C and I have an AuthorizationPolicy that OR on them
services.AddMicrosoftIdentityWebApiAuthentication(
_configuration, "AzureB2C:Tenant1", jwtBearerScheme: "Scheme1");
services.AddMicrosoftIdentityWebApiAuthentication(
_configuration, "AzureB2C:Tenant2", jwtBearerScheme: "Scheme2");public static readonly AuthorizationPolicy Default = new AuthorizationPolicyBuilder()
.AddAuthenticationSchemes(AuthorizationSchemes.Scheme1, AuthorizationSchemes.Scheme2)
.RequireAuthenticatedUser()
.Build();So depending on the order that they get tried, one always fails. Prior to 1.25.1, my App Insight was "clean", i didn't have log output for authentication that, ultimately, are working. Making them non-pertinent. Changing the Log level for the whole thing would actually hide messages that are pertinent, isn't ? Or I wasn't seeing these messages in the past anyways so hiding them thru log levels in 1.25.1 would turn it back to how 1.25.0 behaves, no more, no less ?
Obviously, i may me missing something here and maybe there is a better way to do what we are trying to achieve, that wouldn't try to validate tokens towards the wrong scheme?
I the meantime, 1.25.0 is just fine for now.
jmprieur
commented
2 years ago Oh I see. thanks for the feedback and the explanation. So we are now surfacing too many details, and the ASP.NET core logs were enough in that case.
@sruke, @jennyf19 : Let's see what we can do here to have less noisy logs. Maybe we could have a special namespace (Microsoft.Identity.Model) for the IIdentityLogger so that customers who are not interested in the Microsoft.IdentityModel logs don't get them.
@nlz242, aside:
You should not use AddMicrosoftIdentityWebApiAuthentication in your case as it does both:
AddMicrosoftIdentityWebApi ...So in your code above you set the default authentication scheme twice
what you'd want to do is: service.AddAuthentication("Scheme1) // if you want scheme1 to be the default scheme .AddMicrosoftIdentityWebApi( _configuration, "AzureB2C:Tenant1", jwtBearerScheme: "Scheme1"); service.AddAuthentication() .AddMicrosoftIdentityWebApi( _configuration, "AzureB2C:Tenant2", jwtBearerScheme: "Scheme2");
The "try until it works, or all configured schemes have been tried" is handled by ASP.NET Core's JwtBearer middleware (used internally by Microsoft.Identity.Web.
But in any case that's not going to change anything to your logs issue.
PetteriPertola
commented
1 year ago @jmprieur : do you have any plans to fix this issue? We are facing exactly the same issue as OP and @maffelbaffel in the linked issue above, after upgrading to 1.25.1 or higher. We also have multiple authentication schemes: one for B2C and another using Azure AD for app registration type of flow.
For us this is even worse as our monitoring system depends on the logs being error-free, so our operations team would be overwhelmed by false positives.
To me the framework is not working correctly when it is validating the tokens not belonging to the authentication scheme in question and logs errors that are not real errors.
jmprieur
commented
1 year ago @PetteriPertola did you try with 2.5.0? If this still repro with that one, @jennyf19, I wonder if we shouldn't disable the IdentityLogger? JwtBearer summarizes the issues anyways?
maffelbaffel
commented
1 year ago Not @PetteriPertola but I don't think the logging is the actual problem. Its just a symptom of the tokens being validated with the wrong authentication schema for some reason.
Logging just made the previously hidden problem visible.
For me the problem repoduces with 2.5.0.
damienbod
commented
1 year ago This problem has always existed and still does. As soon as I use multiple identity providers and one is not AAD or AAD B2C, I recommend not using this nuget package because it breaks everything. I default back to JWT or default OIDC depending on the APP type.
PetteriPertola
commented
1 year ago @PetteriPertola did you try with 2.5.0? If this still repro with that one, @jennyf19, I wonder if we shouldn't disable the IdentityLogger? JwtBearer summarizes the issues anyways?
As mentioned by maffelbaffel above, all versions above 1.25.1 have this problem, including 2.5.0.
This leaves us stuck with 1.25.0 and unable to upgrade further.
In our case: External client flow uses B2C, i.e. customers accessing our API. Subsequently, our API (now the client) calls another internal API (the remote server) using the regular AD authentication using app registration setup. The B2C flow seems to hit the Azure AD authentication validation in this case, because there is no way to wire ITokenAquisition into the dependency graph without adding it via the AddMicrosoftIdentityWebApiAuthentication extension method to IServiceCollection:
services.AddMicrosoftIdentityWebApiAuthentication(Configuration).**EnableTokenAcquisitionToCallDownstreamApi**();This step basically tells the Web API that it should use the Azure AD authentication for our controllers when authorizing incoming requests. And since these are authorized using B2C, it will start logging errors whenever it hits the Azure AD authentication part of the request pipeline.
We are only using the ITokenAcquisition after the B2C flow in order to aquire a token for the subsequent downstream call to our internal API. Since it's entirely plausible to have a different authentication/authorization scheme externally than internally, the real solution would be to make it possible to wire the ITokenAcquisition part, without having to simultaneously enable Microsoft Identity Web API Authentication for the entire project.
PetteriPertola
commented
1 year ago @jmprieur @jennyf19 any thoughts on the above?
PetteriPertola
commented
1 year ago @jmprieur @jennyf19 any update on this? Sorry to bother but it is a big issue for us.
mp-jrc
commented
1 year ago Still looking for an update or workaround for this. Seems weird to log all these failures. It even tries to use a scheme that is not accepted for the policy on a specific endpoint, then logs all these errors. Seems weird it even tries.
PetteriPertola
commented
12 months ago @jmprieur @jennyf19 will this be fixed anytime soon?
councilwisenick
commented
12 months ago If it's an option for you, you can get rid of most of the errors by changing your JWTs to use RSA instead. Obviously this will revoke all your current tokens so be aware of that, but the log spam is much more manageable.
I'm down to one error now which is:
[Error] Microsoft.IdentityModel.LoggingExtensions.IdentityLoggerAdapter: IDX40001: Issuer: '...', does not match any of the valid issuers provided for this application.
You could remove that last error by not validating issuer, but I am not willing to do that. I will receive this error every time someone uses one of the tokens for my other auth scheme. It would be fantastic if tokens meant for the other scheme would not generate these errors.
PetteriPertola
commented
11 months ago @councilwisenick thanks for the suggestion, however any line that is an error would flare up an alert in our monitoring systems so that would be non-stop incidents as a result.
nlz242
commented
11 months ago @PetteriPertola did you try with 2.5.0? If this still repro with that one, @jennyf19, I wonder if we shouldn't disable the IdentityLogger? JwtBearer summarizes the issues anyways?
This is what we did. We basically created a "null logger"
public class NullIdentityLogger : IIdentityLogger
{
public bool IsEnabled(EventLogLevel eventLogLevel)
{
return false;
}
public void Log(LogEntry entry)
{
}
}And then asigned it as the logger :
LogHelper.Logger = new NullIdentityLogger();In our case, not only was the IdentityLogger logging way too much stuff, as soon as it got into the logger it would tank performance and add >1s per call, leading to very poor system performance.
Hopefuly the team end up taking this problem seriously and find a satisfactory resolution, but it's been a minute already.
By the way, if you use this solution.... DO NOT use the NullIdentityModelLogger provided in Microsoft.IdentityModel.Abstractions... Microsoft.Identity.Web actively tests for it and will REPLACE IT if you are using it... thus bringing back the issue you were trying to solve in the first place... i created an issue for that here https://github.com/AzureAD/microsoft-identity-web/issues/2675
erionpc
commented
9 months ago I had the same problem, and I found a very easy solution, which does the job. Just add this to config.
"Logging": {
"LogLevel": {
"Microsoft.IdentityModel.Logging": "None"
}
}It's kind of a bad thing to suppress logs like this... but using inspiration from @erionpc above, we did the same for SeriLog... However this will obviously not log any other errors or other information from this specific class. I still think microsoft devs should fix this as it's very common to have e.g. B2C token in azure "incoming" and then the Identity Web style Azure AD token when calling a downstream service...
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.MinimumLevel.Override("Microsoft.IdentityModel.LoggingExtensions.IdentityLoggerAdapter", LogEventLevel.Fatal)Updating to Microsoft.Identity.Web v2.19.0 (as part of dotnet8 update) changes the error log message prefix to IDX10517:
IDX10517: Signature validation failed. The token's kid is missing. Keys tried: 'Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'V1Y09OFMBXAZUFfJ-vWjlrH-ZSc', InternalId: 'V1Y09OFMBXAZUFfJ-vWjlrH-ZSc'. , KeyId: V1Y09OFMBXAZUFfJ-vWjlrH-ZSc Microsoft.IdentityModel.Tokens.RsaSecurityKey, KeyId: 'V1Y09OFMBXAZUFfJ-vWjlrH-ZSc', InternalId: '21yYN7-15_HEG6Myr8mw1D07vAMCN503j4n1GnsWWJQ'. , KeyId: V1Y09OFMBXAZUFfJ-vWjlrH-ZSc Microsoft.IdentityModel.Tokens.X509SecurityKey, KeyId: 'L1KfKFI_jnXbwWc22xZxw1sUHH0', InternalId: 'L1KfKFI_jnXbwWc22xZxw1sUHH0'. , KeyId: L1KfKFI_jnXbwWc22xZxw1sUHH0 Microsoft.IdentityModel.Tokens.RsaSecurityKey, KeyId: 'L1KfKFI_jnXbwWc22xZxw1sUHH0', InternalId:...I guess we're stuck using the various log filtering mechanisms to exclude these messages until someone at Microsoft decides to fix this properly :(. At the very least the messages should be turned down to warnings. It's hard to see why someone decided it's an error condition for a token validator, that can exist alongside other validators, should generate an error for tokens it dones't own.
"Serilog": {
"Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.Seq", "Serilog.Expressions" ],
"WriteTo": [
{
"Name": "Console",
"Args": {
//"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff} [{Level:u3}] {Message:lj} ({SourceContext}){NewLine}{Exception}"
}
},
{
"Name": "Seq",
"Args": {
"serverUrl": "http://localhost:5341",
"apiKey": "UqHK2h3G69XYpZ4t1ZWc"
}
}
],
"Filter": [
{
"Name": "ByExcluding",
"Args": {
"expression": "StartsWith(@m, 'IDX40003:') or StartsWith(@m, 'IDX10205:') or StartsWith(@m, 'IDX10233:') or StartsWith(@m, 'IDX10235:') or StartsWith(@m, 'IDX10503:') or StartsWith(@m, 'IDX10634:' or StartsWith(@m, 'IDX10517:') or StartsWith(@m, 'IDX10720:'))"
}
}
],
"MinimumLevel": {
"Default": "Debug",
"Override": {
"Microsoft": "Warning",
"Microsoft.EntityFrameworkCore": "Warning",
"Microsoft.Hosting.Lifetime": "Information",
"MassTransit.Messages": "Information",
"Microsoft.AspNetCore.Authentication": "Warning",
"NoFrixion.MoneyMoov.Api.Middleware.Authorization.ApiKeyAuthorisationHandler": "Information"
//"Microsoft.AspNetCore.HttpLogging": "Verbose"
}
}
},
Microsoft.Identity.Web Library
Microsoft.Identity.Web
Microsoft.Identity.Web version
1.25.1
Web app
Sign-in users and call web APIs
Web API
Protected web APIs (validating tokens)
Token cache serialization
Not Applicable
Description
I'm attempting to use JWT tokens from 3 different sources:
The first two are able to co-exist happily. As soon as Azure AD is added via
AddMicrosoftIdentityWebApiorAddMicrosoftIdentityWebApiAuthenticationthe MSAL library attempts to validate all tokens and there does not seem to be a way to specify that it should only validate tokens from Azure AD.Reproduction steps
ConfigureServicesadd multiple JWT authentication providers, as below. This example shows 3 but the issue can be demonstrated with only one. The purpose of having 3 is to show that the 2 non Azure AD JWT sources are able to co-exist.Error message
When attempting to use a custom JWT token the following exception messages are generated by MSAL. The token does then get verified by the custom JWT token validation.
Relevant code snippets
Regression
No response
Expected behavior
The MSAL library should ignore any JWT tokens that have not been issued by Azure AD if there are muliple JWT issuers configured..