Azure / azure-cosmos-dotnet-v3

.NET SDK for Azure Cosmos DB for the core SQL API
MIT License
738 stars 493 forks source link

Account key "not a valid base-64 string" #4764

Closed ShmuelCammebys closed 3 days ago

ShmuelCammebys commented 1 week ago

Describe the bug When passing either the exact connection string from the Azure Portal (i.e. new CosmosClient("AccountEndpoint=...;AccountKey=...;")) or separately passing the token and the URI (i.e. new CosmosClient("https://...", new AzureKeyCredential("<base64 string>")), the SDK throws the error: System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

To Reproduce Create Azure Functions template (isolated workers .NET 8.0) in Visual Studio, and add this to Program.cs

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(services =>
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();

        services.AddSingleton<CosmosClient>(provider => new CosmosClient(...));
    })
    .Build();

Expected behavior CosmosClient created without an issue.

Actual behavior Error is thrown.

Environment summary SDK Version: 3.43.1 OS Version: MacOSX (Sequoia 15.0)

Additional context Full stacktrace (app specific calls removed):

Exception: System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
[2024-10-08T04:38:45.539Z]    at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
[2024-10-08T04:38:45.539Z]    at System.Convert.FromBase64String(String s)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.StringHMACSHA256Hash..ctor(String base64EncodedKey)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.AuthorizationTokenProviderMasterKey..ctor(String authKey)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.AuthorizationTokenProvider.CreateWithResourceTokenOrAuthKey(String authKeyOrResourceToken)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.CosmosClient..ctor(String accountEndpoint, String authKeyOrResourceToken, CosmosClientOptions clientOptions)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.SingletonCosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateQuery(String containerId, String partitionKey, CosmosSqlQuery query)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.DocumentAsyncEnumerable.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Query.Internal.CosmosShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.539Z]    at MyApp.DirectFunctionExecutor.ExecuteAsync(FunctionContext context) in /Users/work/RiderProjects/MyApp/MyApp/Microsoft.Azure.Functions.Worker.Sdk.Generators/Microsoft.Azure.Functions.Worker.Sdk.Generators.FunctionExecutorGenerator/GeneratedFunctionExecutor.g.cs:line 55
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\_work\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 13
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in /mnt/vss/_work/1/s/extensions/Worker.Extensions.Http.AspNetCore/src/FunctionsMiddleware/FunctionsHttpProxyingMiddleware.cs:line 54
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in D:\a\_work\1\s\src\DotNetWorker.Core\FunctionsApplication.cs:line 91
[2024-10-08T04:38:45.539Z] Stack:    at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
[2024-10-08T04:38:45.539Z]    at System.Convert.FromBase64String(String s)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.StringHMACSHA256Hash..ctor(String base64EncodedKey)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.AuthorizationTokenProviderMasterKey..ctor(String authKey)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.AuthorizationTokenProvider.CreateWithResourceTokenOrAuthKey(String authKeyOrResourceToken)
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Cosmos.CosmosClient..ctor(String accountEndpoint, String authKeyOrResourceToken, CosmosClientOptions clientOptions)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.SingletonCosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateQuery(String containerId, String partitionKey, CosmosSqlQuery query)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.DocumentAsyncEnumerable.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Cosmos.Query.Internal.CosmosShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.539Z]    at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.539Z]    at MyApp.DirectFunctionExecutor.ExecuteAsync(FunctionContext context) in /Users/work/RiderProjects/MyApp/MyApp/Microsoft.Azure.Functions.Worker.Sdk.Generators/Microsoft.Azure.Functions.Worker.Sdk.Generators.FunctionExecutorGenerator/GeneratedFunctionExecutor.g.cs:line 55
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\_work\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 13
[2024-10-08T04:38:45.539Z]    at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in /mnt/vss/_work/1/s/extensions/Worker.Extensions.Http.AspNetCore/src/FunctionsMiddleware/FunctionsHttpProxyingMiddleware.cs:line 54
[2024-10-08T04:38:45.540Z]    at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in D:\a\_work\1\s\src\DotNetWorker.Core\FunctionsApplication.cs:line 91.
[2024-10-08T04:38:45.569Z] fail: Microsoft.Azure.Functions.Worker.FunctionsApplication[2]
[2024-10-08T04:38:45.569Z]       Function 'MyFunction', Invocation id '35befd39-f012-42a2-8792-cd0f88ef4f4c': An exception was thrown by the invocation.
[2024-10-08T04:38:45.569Z]       System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
[2024-10-08T04:38:45.569Z]          at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
[2024-10-08T04:38:45.569Z]          at System.Convert.FromBase64String(String s)
[2024-10-08T04:38:45.569Z]          at Microsoft.Azure.Cosmos.StringHMACSHA256Hash..ctor(String base64EncodedKey)
[2024-10-08T04:38:45.569Z]          at Microsoft.Azure.Cosmos.AuthorizationTokenProviderMasterKey..ctor(String authKey)
[2024-10-08T04:38:45.569Z]          at Microsoft.Azure.Cosmos.AuthorizationTokenProvider.CreateWithResourceTokenOrAuthKey(String authKeyOrResourceToken)
[2024-10-08T04:38:45.569Z]          at Microsoft.Azure.Cosmos.CosmosClient..ctor(String accountEndpoint, String authKeyOrResourceToken, CosmosClientOptions clientOptions)
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.SingletonCosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.get_Client()
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateQuery(String containerId, String partitionKey, CosmosSqlQuery query)
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.DocumentAsyncEnumerable.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Cosmos.Query.Internal.CosmosShapedQueryCompilingExpressionVisitor.QueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.569Z]          at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.SingleOrDefaultAsync[TSource](IAsyncEnumerable`1 asyncEnumerable, CancellationToken cancellationToken)
[2024-10-08T04:38:45.570Z]          at MyApp.DirectFunctionExecutor.ExecuteAsync(FunctionContext context) in /Users/work/RiderProjects/MyApp/MyApp/Microsoft.Azure.Functions.Worker.Sdk.Generators/Microsoft.Azure.Functions.Worker.Sdk.Generators.FunctionExecutorGenerator/GeneratedFunctionExecutor.g.cs:line 55
[2024-10-08T04:38:45.570Z]          at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\_work\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 13
[2024-10-08T04:38:45.570Z]          at Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore.FunctionsHttpProxyingMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in /mnt/vss/_work/1/s/extensions/Worker.Extensions.Http.AspNetCore/src/FunctionsMiddleware/FunctionsHttpProxyingMiddleware.cs:line 54
[2024-10-08T04:38:45.570Z]          at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in D:\a\_work\1\s\src\DotNetWorker.Core\FunctionsApplication.cs:line 91
philipthomas-MSFT commented 6 days ago

Hello @ShmuelCammebys,

Thank you for reaching out to us today! I’d be happy to assist you with your issue. To better understand what might be happening, could you please provide a code snippet or more details on your setup?

A few additional questions to help troubleshoot:

The more details you can share, the better we can pinpoint the issue and find a solution. Looking forward to hearing back from you!

ShmuelCammebys commented 6 days ago
  1. I have never used the cosmos ask
  2. I have not programmatically connected to cosmos before.
  3. I have not tried a simpler setup.
  4. See original post for logs.
philipthomas-MSFT commented 6 days ago

could you please provide a code snippet or more details on your setup

Hello @ShmuelCammebys,

Thanks for your reply. I'm trying to gather more information from you so I can better assist. I previously requested code snippets, but I’d like to walk through the issue again.

You mentioned that you're copying a connection string from the Portal and also trying to use AzureKeyCredential.

Let’s think through this for a minute. You mentioned you're receiving a FormatException with the message:

"The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters."

This happens irrespective of whether you use a connection string or a credential, correct?

To help you further, I'd recommend checking out our documentation here if you haven't already.

Now, here's where I’d like to step in. I genuinely want to assist because your success is our success. But I need more information to move forward. How can I reproduce this issue? Have you tried building a simple application, as I suggested, using the documentation link above? Are you certain you're getting the Base-64 FormatException with both approaches?

Please provide details on how to reproduce this issue, and let’s work together to identify the gap and resolve it.

philipthomas-MSFT commented 3 days ago

Hello @ShmuelCammebys,

I will go ahead and close this issue for the time being. However, please don’t hesitate to reach out if you have any further questions or require additional assistance. I sincerely hope that the documentation we provided brings you closer to mitigating and ultimately resolving your issue.

Thank you once again for your continued support and for choosing to use our product. We greatly value your trust in us.