Missing headers/preflight `OPTIONS` request in amazonka-2.0

[9999years]

I'm attempting to migrate our codebase to using amazonka-2.0. However, our AWS API calls no longer succeed. I've done a diff on the requests with the two library versions and I've found that amazonka-2.0 isn't setting the following headers on my requests:

Connection: keep-alive
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site

Also, amazonka-1.6 would make a "preflight" OPTIONS request before the PUT request, with the aforementioned headers as well as a Access-Control-Request-Method: PUT header.

Finally, amazonka-1.6 is making requests to s3.amazonaws.com/bucket-name/..., while amazonka-2.0 makes requests to bucket-name.s3.region.amazonaws.com/.... EDIT: I found the logic for this here: https://github.com/brendanhay/amazonka/blob/3e66cc1b07f2818d0b4fc346310ecdb1c3c27599/lib/amazonka-core/src/Amazonka/Request.hs#L160-L165

I'm not sure where these discrepancies are coming from – most likely, I'm missing an amazonka-2.0 API I need to be using. Do you have any ideas? I've checked the amazonka changelog and grepped the logs in our codebase as well as the amazonka codebase to try to find where these headers might have been set, but I can't find anything.

[endgame]

I'd like to be able to help, but it is very hard for me to diagnose this on the information you've given so far. What S3 APIs are you calling, why did you need a preflight OPTIONS, what are the additional headers for, and are you talking to real S3 or a "compatible" object store?

The bucketname.s3.amazonaws.com change is the modern "vhost addressing style" used by modern versions of the official SDK. If you need to, you can overwrite it by overriding the s3AddressingStyle field on S3:

https://github.com/brendanhay/amazonka/blob/3e66cc1b07f2818d0b4fc346310ecdb1c3c27599/lib/amazonka-core/src/Amazonka/Types.hs#L519

[brendanhay]

Regarding CORS - none of those headers were explicitly set by amazonka-1.6, they'd necessarily have to be either configured for the http-client Manager that is implicitly created or passed to amazonka's Env (see also newEnvWith), which would typically be where you'd add these extra headers.

Are you able to diff/blame how Env is created and any http-client Manager modifications that occur in the old vs new code?

[9999years]

Thanks, that's very helpful. I'll check out the Env and see if I can track it down.

[9999years]

Unfortunately, setting S3AddressingStylePath in the overrides didn't help. It looks like in the particular case I'm testing, presignURL is used to generate a signed URL which is passed to the frontend, and then the frontend makes the actual PUT request to S3. I'm not sure why the behavior would change in the frontend, but the amazonka code is all I'm changing in the backend...

[9999years]

Well, maybe not; even with S3AddressingStylePath, amazonka-2.0 generates s3.us-east-1.amazonaws.com/bucket-name/... rather than s3.amazonaws.com/bucket-name/....

[9999years]

Oh, in case it's relevant: the requests that the frontend makes show up in the browser debug console as blocked by the content security policy.

I haven't gotten anywhere with this so I think I'm going to shelve this upgrade attempt for now. I don't mind if you close this issue because it seems like it might be a configuration issue somewhere on our end, but do remain very curious about why this happened — as I mentioned, the only thing I changed was the amazonka library, so I'm not sure why the frontend would suddenly start failing.

[endgame]

If presigning is generating S3 URLs in the wrong addressing style then that's a legitimate bug. Are you able to provide a minimal example of this?

And if you could provide cut-down examples of your program (including frontend) that demonstrate the behaviour change between 1.6 and 2.0 main, that would also be extremely helpful.

[9999years]

Okay, it looks like in the frontend we have a content security policy that specifically whitelists domains, and s3.us-east-1.amazonaws.com or the virtual host addressing style isn't included in those. Now we just have to track down where that URL is getting generated.

[9999years]

A-ha! Here, in amazonka-1.6, the endpoint for S3 in us-east-1 is special cased to s3.amazonaws.com:

https://github.com/brendanhay/amazonka/blob/6bc20b0134590152f6b3abe247d13c480c251495/core/src/Network/AWS/Endpoint.hs#L56-L59

On the current main, there's no special case for S3 or us-east-1, so it gets translated to s3.us-east-1.amazonaws.com:

https://github.com/brendanhay/amazonka/blob/3e66cc1b07f2818d0b4fc346310ecdb1c3c27599/lib/amazonka-core/src/Amazonka/Endpoint.hs#L70-L72

I think we can close this — I'll update our content security policy to match and then we should be golden. Thanks for the help!