Setting S3 endpoint creates invalid host in HTTP request

[chreekat]

Repro:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Amazonka
import Amazonka.Auth
import Amazonka.S3

main :: IO ()
main = do
    env <- do
        e <- newEnv (pure . fromKeys "key" "secret")
        let s3 = Amazonka.setEndpoint True "my-cool-website" 443 Amazonka.S3.defaultService
        pure $ Amazonka.configureService s3 e
        --pure e
    let req = newListObjectsV2 "my-cool-bucket"
    print req
    resp <- runResourceT $ send env req
    print resp

Result of cabal run:

ListObjectsV2' {continuationToken = Nothing, delimiter = Nothing, encodingType = Nothing, expectedBucketOwner = Nothing, fetchOwner = Nothing, maxKeys = Nothing, prefix = Nothing, requestPayer = Nothing, startAfter = Nothing, bucket = BucketName {fromBucketName = "my-cool-bucket"}}
repro: TransportError (HttpExceptionRequest Request {
  host                 = "my-cool-bucket.my-cool-website"
  port                 = 443
  secure               = True
  requestHeaders       = [("X-Amz-Content-SHA256","e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"),("X-Amz-Date","20240228T165613Z"),("Host","my-cool-bucket.my-cool-website"),("Authorization","<REDACTED>")]
  path                 = "/"
  queryString          = "?list-type=2"
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 0
  responseTimeout      = ResponseTimeoutMicro 70000000
  requestVersion       = HTTP/1.1
  proxySecureMode      = ProxySecureWithConnect
}

The host header looks pretty wrong! Is this possibly a bug in amazonka?

• amazonka-2.0
• amazonka-s3-2.0

[endgame]

I think to solve your immediate problem, you can set the S3 addressing style when you override the Service.

Can you try a similar request with an official SDK and see what it does? Official SDKs might be a bit better about deciding when to use vhost addressing style (in which case we should copy some of their logic), or they might make a similar mistake (in which case behaving like other SDKs is probably more important than adding magic).

[chreekat]

I discovered I was using https://hostname as my endpoint when I should have just used hostname.

I simply expected to see the bucket name in the url rather than in the hostname, though I'm not sure why.. So that is what caught my eye in the exception message. It was especially noticeable when the line read host = "my-cool-bucket.https://my-cool-hostname, which is clearly nonsense.

So I suppose there is no bug here, after all. Though as a feature it might be nice if including the protocol was allowed, since it's actually required in the AWS_ENDPOINT_URL environment variable.

[chreekat]

I'll close this in the meanwhile.

[endgame]

Do we even want to respect AWS_ENDPOINT_URL? I see it in https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html , which is more of a CLI config option than an SDK config option.

[chreekat]

Maybe there's some distinction you know better than I do. For me, it's grouped in with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as "things I need to set on a per-environment basis".

[endgame]

Seems like it's supported by the SDK too: https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html

I've made a separate issue: #980