Issue when Using Continuation Token

[willfore]

I'll start by saying this could totally be something I'm missing but I'm not sure what. I'm attempting to query the next results using the continuation token and it is returning SignatureDoesNotMatch. I've double checked all of the attempts using bruno and everything works just fine. Here's the details:

Make initial S3 query:

req = Req.new() |> ReqS3.attach()
Req.get!(req, url: "s3://bucket/?list-type=2&max-keys=1")

Response:

%{
   "Contents" => [
     %{
       "ETag" => "\"d41d8cd98f00b204e9800998ecf8427e\"",
       "Key" => "bucket/somefile.txt",
       "LastModified" => "2024-07-29T20:43:28Z",
       "Owner" => %{"ID" => nil},
       "Size" => "12345",
       "StorageClass" => "STANDARD"
     }
   ],
   "ContinuationToken" => nil,
   "IsTruncated" => "true",
   "KeyCount" => "1",
   "MaxKeys" => "1",
   "Name" => "bucket",
   "NextContinuationToken" => "bucket/somefile2.txt",
   "Prefix" => nil
 }}

I then setup my next query

Req.get!(req, url: "s3://bucket/?list-type=2&max-keys=1&continuation-token=bucket/somefile2.txt")

This always returns a 403 status with the code SignatureDoesNotMatch. When i do the same exact attempt using Bruno or Postman or even curl it works just fine. After debugging the request I don't see anything going on different that would cause the issue but admittedly I may have missed something. I thought it may be the additional parameters at first but that seems to work fine on the initial requests without issue. Any ideas on what it may be?

[wojtekmach]

Thank you for the report. Are you using the latest version? We have a custom aws signature implementation in req and there is probably a bug. Could you try using Req without :aws_sigv4 option and do the signature using :aws_signature dep?

[willfore]

I'll give that a shot and report back shortly. Thanks!

[willfore]

So I'm using req 0.5.6 and req_s3 0.2.1. For the auth portion I simply set the env vars as suggested in the docs. I didn't see any mention of a aws_signature dep. Can you point me in that direction?

[wojtekmach]

Yeah sorry I meant as if the :aws_sigv4 option and req_s3 didn’t exist. You can use aws_signature to sign a request and req to make it: https://gist.github.com/wojtekmach/90efd7bbec164895629704362cf4a130.

[willfore]

so using the snippet you gave as an example I can run the initial calls with no issue but when I attempt to utilize the continuation-token I run into the same error.

[wojtekmach]

Could you try passing uri_encode_path: false as the last argument to aws_signature?

[willfore]

Same issue unfortunately

[willfore]

Just to update ( not that I want to use it ) if I do these same calls using the ex_aws libraries it works without issue.