brendanhay / amazonka

A comprehensive Amazon Web Services SDK for Haskell.
https://amazonka.brendanhay.nz
Other
605 stars 226 forks source link

sagemaker-runtime InvokeEndpoint double-encodes the body #1010

Open pbrisbin opened 3 days ago

pbrisbin commented 3 days ago

We're using the InvokeEndpoint function with JSON as the body. We believe it arrives at the server double-encoded. I say "we believe" because it's really hard to log what we're seeing and be confident we aren't introducing our own escaping :sweat_smile: .

Anyway, I think this is because of the use of postJSON and toJSON body (where body :: Text and contains already-encoded JSON in our case):

instance Core.AWSRequest InvokeEndpoint where
  request overrides =
    Request.postJSON (overrides defaultService)

instance Data.ToJSON InvokeEndpoint where
  toJSON InvokeEndpoint' {..} = Data.toJSON body

From the docs, the entire body is meant to be included as itself and is assumed to be in the given Content-Type.

And, as far as I can tell (I find searching for this stuff difficult), the CLI/python libraries do this:

        r = self.http.request("POST", url, body=Body, preload_content=False, headers=headers)

source

So I think the body field should probably be ByteString and sent to the server directly, not via postJSON.

pbrisbin commented 2 days ago

From those same docs, it looks like the response can be any content-type as well. So we shouldn't be using receiveJSON here either, I'd think.