search

nginxinc / nginx-s3-gateway

NGINX S3 Caching Gateway
Apache License 2.0
522 stars 129 forks source link

The request signature we calculated does not match the signature #251

Open varontron opened 6 months ago

varontron commented 6 months ago

Describe the bug

Receive "The request signature we calculated does not match the signature" error when attempting to load a resource from s3, with proxy_intercept_errors off

To reproduce

Steps to reproduce the behavior:

  1. Set env var AWS_SIGS_VERSION=4
  2. Launch container
  3. Attempt to load resource in browser
  4. See error

Expected behavior

Expect resource to load

Your environment

Additional context

System was working flawlessly for months using AWS_SIGS_VERSION=4, launched with docker-compose. A couple days ago, I added an additional contaner spec to docker-compose.yml and restarted. The problem seemed to arise when a new version of the gateway image was downloaded and deployed.

Changing the config to use AWS_SIGS_VERSION=2 re-enables the system and delivers the resources as expected.

Regrettably I don't have a handy backup of the docker-compose.yml config, but I'm reasonably confident the AWS_SIGS_VERSION was not changed from 2 to 4 prior to the restart.

Is it possible something in the latest image broke with regard to the AWS_SIGS_VERSION setting?

Sensitive Information

Remember to redact any sensitive information such as authentication credentials or license keys.

4141done commented 6 months ago

Thank you for your report, @varontron and I'm sorry that you are having trouble. I can't reproduce this issue myself, but we did ship a change recently that could have influenced the signature calculation.

A few next steps:

  1. Can you see if this tag: latest-njs-oss-20240306 Which is the one right before that change fixes your issue? If it does that will give us a start.

  2. Try with the latest image is specifying these two config options:

S3_STYLE=virtual-v2
S3_SERVICE=s3

I am attempting to migrate us to S3's current recommendations on pathing. I made every effort to keep the defaults the same but it's possible a bug snuck through my testing.

  1. Can you supply a copy of your config settings with sensitive information redacted? That could help me with a repro.
dekobon commented 6 months ago

Another factor that can cause this is clock skew. I would ensure that your compute instance running the gateway is synced with a ntp server.

spijet commented 3 months ago

I'm having a similar issue when using the unprivileged-oss image to serve files off a private bucket in MinIO. Switching to deprecated AWS signature version 2 helped me as well. Here's how the environment looks like in my case:


AWS_ACCESS_KEY_ID=<redacted>
AWS_SECRET_ACCESS_KEY=<redacted>
AWS_SIGS_VERSION=2
S3_BUCKET_NAME=<redacted>
S3_REGION=us-east-1
S3_SERVER=minio-headless.minio.svc.cluster.local
S3_SERVER_PORT=9000
S3_SERVER_PROTO=http
S3_STYLE=path
4141done commented 2 months ago

@spijet Thank you for the report, although I haven't been able to determine the cause of this issue yet each data point helps. Out of curiosity, did this issue come up recently after you'd been using the signature V4 successfully? I have a suspicion that a recent change could have introduced a bug for some use cases but I have not been able to reproduce the issue on any of my test setups.