AwsSigV4 signing for Amazon OpenSearch Serverless not yet released to NuGet

[phillip-proulx]

What is the bug?

The changes made in approved PR #133 have not yet been released to NuGet.

How can one reproduce the bug?

You can see that these changes failed to create the NuGet artifact here: https://github.com/opensearch-project/opensearch-net/actions/runs/3970496588/jobs/6806292731

Furthermore, the latest version of the NuGet package was updated 4 months ago while the PR was merged in 2 months ago: https://www.nuget.org/packages/OpenSearch.Net.Auth.AwsSigV4/1.2.0

What is the expected behavior?

The changes committed on PR #133 should be available in NuGet.

What is your host/environment?

.NET Core running .NET 6

Do you have any screenshots?

Do you have any additional context?

N/A

[Xtansia]

Hi @phillip-proulx,

The workflow you've linked to is purely tests & validates a local .nupkg file, it does not perform any releasing itself. The name is a little misleading though I'll acknowledge.

We will be attempting to get out a release very soon and I'll update this issue once it's available.

[phillip-proulx]

Hi @Xtansia ,

Thank you for the update!

Do you have a ETA on that release?

I would like to schedule my current development around it.

[Xtansia]

There's a chance it'll happen today if all goes to plan, otherwise within the next week

[turacma]

Not sure if this is the most appropriate place to mention these, but we stumbled upon this issue while diving into understanding why this client wasn't working when connecting to OpenSearch Serverless. Issues we noted:

1. The documentation is already updated to reflect this update, even though it hasn't been released. This lead to some confusion where we thought the change to specify the service name had been deprecated rather just unreleased.
2. The actual AOSS service is returning an incorrect error message indicating the signature needs to be scoped to es not the actual correct value aoss.

Anyway, looking forward to seeing the release so we can actually use this library with OpenSearch Serverless, but some things to consider looking at in your development workflows.

[wbeckler]

2. The actual AOSS service is returning an incorrect error message indicating the signature needs to be scoped to es not the actual correct value aoss.

Can you tell if this is an issue with the client expressing the error, or is it in the verbiage from Server less?

[maghis]

@wbeckler we are pretty confident the error is in the message returned from Serverless. The exact error we are getting is: {Type: AccessDenied Reason: "Credential should be scoped to correct service: 'es'."}

[phillip-proulx]

@wbeckler I agree with @maghis, I believe this is from AWS. You can see an example of this error message under the "Service" section of the SigV4 troubleshoot documentation: https://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html

[dblock]

1. The documentation is already updated to reflect this update, even though it hasn't been released. This lead to some confusion where we thought the change to specify the service name had been deprecated rather just unreleased.

I think we need a disclaimer in USER_GUIDE/README similar to https://github.com/ruby-grape/grape#stable-release, want to PR?

[dblock]

@wbeckler I agree with @maghis, I believe this is from AWS. You can see an example of this error message under the "Service" section of the SigV4 troubleshoot documentation: https://docs.aws.amazon.com/general/latest/gr/signature-v4-troubleshooting.html

I think we're misinterpreting the message. It doesn't tell you what service to use (aoss), it tells you that you're using an invalid value (es).

$ awscurl "$ENDPOINT/_cat/indices" --region $AWS_REGION --service foobar
{"status":403,"request-id":"e306a61e-5812-94ce-854c-5b9d909f3c27","error":{"reason":"Credential should be scoped to correct service: 'foobar'.","type":"AccessDenied"}}

I'll followup to see whether this is by design or whether this is something the service can/should change. I think the error should be:

Credential should be scoped to correct service: 'aoss', not 'foobar'.

[turacma]

I think we're misinterpreting the message. It doesn't tell you what service to use (aoss), it tells you that you're using an invalid value (es).

@dblock I used to work for multiple AWS service teams, I'm fairly confident this message is supposed to tell you the correct service name. It is typically generated by the common library service teams use to validate signatures. My suspicion is that the OpenSearch service is wrapping that error and returning a hardcoded value which is incorrect for the Serverless version. My mistake, missed the end of your message, clearly it's overriding the default behavior of the validation library or using one I'm not familiar with.

[turacma]

I think we need a disclaimer in USER_GUIDE/README similar to https://github.com/ruby-grape/grape#stable-release, want to PR?

Done: https://github.com/opensearch-project/documentation-website/pull/3428

[turacma]

@dblock Here's an example of what I would expect from a service I used to work on:

% awscurl https://apigateway.us-east-1.amazonaws.com/restapis --service foobar
{"logref":"b0f1edd0-22ab-48eb-bec4-582cae4ed147","message":"Credential should be scoped to correct service: 'apigateway'. "}

[dblock]

100% @turacma I already submitted a fix

[phillip-proulx]

Hi @Xtansia and @dblock ,

Any updates on the publish timeline?

Thanks!

[Xtansia]

@phillip-proulx We're still aiming to get it out this week, just having to work through some wrinkles with a new release workflow.

[Xtansia]

@phillip-proulx @turacma Version 1.3.0 of the client and AwsSigV4 signer has now been released which includes the support for Amazon OpenSearch Serverless https://www.nuget.org/profiles/opensearchproject

[phillip-proulx]

Thanks @Xtansia!

I appreciate your help on this!

[dblock]

~I added support for OpenSearch Serverless in this sample: https://github.com/dblock/opensearch-node-client-demo/commit/afce84cda37df012b02cd2b66169719a069ef2d4~

[Xtansia]

@dblock Looks like you've linked to the wrong repo

[dblock]

Whoops, I meant https://github.com/dblock/opensearch-dotnet-client-demo/commit/ae38b405c6d7d1cde924fd4394864b303739d826 and https://github.com/dblock/opensearch-dotnet-client-demo/commit/3ca18140a49dc720891a4940be9b1d3aa8b11770