ex-aws / ex_aws_s3

170 stars 147 forks source link

ExAws.S3.presigned_url yields mangled URLs #245

Closed dealloc closed 8 months ago

dealloc commented 8 months ago

Environment

Current behavior

We're generating a presigned URL for an S3 object using following code:

  def get_download_url(%URI{scheme: "s3", path: uuid}) do
    :s3
    |> ExAws.Config.new([])
    |> ExAws.S3.presigned_url(:get, bucket_name(), uuid, [])
  end

However in production we get URL's returned like this: ://httpsfly.storage.tigris.dev/...

Running Application.get_all_env(:ex_aws) in PRD yields the following:

[
  debug_requests: true,
  secret_access_key: {:system, "AWS_SECRET_ACCESS_KEY"},
  json_codec: Jason,
  s3: [
    scheme: "https",
    host: "fly.storage.tigris.dev",
    region: "auto",
    bucket: "<<REDACTED>>"
  ],
  access_key_id: {:system, "AWS_ACCESS_KEY_ID"}
]

Expected behavior

I would expect our URL's to be properly formatted, eg. https://fly.storage.tigris.dev/...

dealloc commented 8 months ago

Okay, so it's partially confusion on my part and partially what I think should be changed (though it would be a breaking change in the library).

Our cloud provider set the URL of the S3 endpoint as a full url (eg. https://fly.storage.tigris.dev) and we split it into the required S3 configuration as following:

  s3_endpoint = "AWS_ENDPOINT_URL_S3"
  |> System.get_env("https://fly.storage.tigris.dev")
  |> URI.parse()

  config :ex_aws, :s3,
    scheme: s3_endpoint.scheme,
    host: s3_endpoint.host,
    region: System.get_env("AWS_REGION", "auto"),
    bucket: System.get_env("BUCKET_NAME")

However, if you use URI's scheme attribute it yields 'https' while ex_aws expects it to be 'https://'

Configuring this wrong yields mangled URLs and it's VERY unclear what went wrong unless you dive in the source code where it builds the signed URL and find this bit

"#{config[:scheme]}#{config[:host]}#{port}/#{bucket}#{object}"

note the absence of slashes between scheme and host. My confusion was that the config is called scheme but is formatted very differently than the default scheme as interpreted in the context of an URI.