PutObject request failing on MinIO because of malformed XML

[basvandijk]

I'm in the process of upgrading my amazonka libraries from 23495aaf7f08d6666f8e3bc8e72f5041ce7c6801 to the latest master (0d70474d888c5e5a43417ac9f7b408e7890ff853).

Unfortunately my integration tests are now failing which test my applications agains a MinIO service simulating AWS S3.

Note that I also upgraded MinIO to its latest RELEASE.2022-07-08T00-05-23Z.

My test is failing when executing the following code:

do env <- Amazonka.newEnv Amazonka.Auth.fromKeysEnv
   ...
   _resp <-
     Amazonka.runResourceT $
       Amazonka.send env $
         S3.newPutObject bucket objectKey rqBody

The send is throwing the exception:

ServiceError
  ( ServiceError'
      { _serviceErrorAbbrev = Abbrev {fromAbbrev = "S3"},
        _serviceErrorStatus = Status {statusCode = 400, statusMessage = "Bad Request"},
        _serviceErrorHeaders =
          [ ("Accept-Ranges", "bytes"),
            ("Content-Length", "481"),
            ("Content-Security-Policy", "block-all-mixed-content"),
            ("Content-Type", "application/xml"),
            ("Server", "MinIO"),
            ("Strict-Transport-Security", "max-age=31536000; includeSubDomains"),
            ("Vary", "Origin"),
            ("Vary", "Accept-Encoding"),
            ("X-Amz-Bucket-Region", "us-east-1"),
            ("X-Amz-Request-Id", "16FFF7C31FFCC336"),
            ("X-Content-Type-Options", "nosniff"),
            ("X-Xss-Protection", "1; mode=block"),
            ("Date", "Fri, 08 Jul 2022 21:26:45 GMT")
          ],
        _serviceErrorCode = ErrorCode "MalformedXML",
        _serviceErrorMessage =
          Just
            ( ErrorMessage
                { fromErrorMessage = "The XML you provided was not well-formed or did not validate against our published schema."
                }
            ),
        _serviceErrorRequestId = Just (RequestId {fromRequestId = "16FFF7C31FFCC336"})
      }
  )

I also see the following corresponding error logged by minio:

{
  "deploymentid": "ef71e5aa-c9bc-48b7-a797-61b2cd01486b",
  "level": "ERROR",
  "errKind": "ALL",
  "time": "2022-07-08T21:26:45.328780889Z",
  "api": {
    "name": "SYSTEM",
    "args": {}
  },
  "error": {
    "message": "XML syntax error on line 2: invalid character entity &��k���J��1���ǹ����zf (no semicolon) (*xml.SyntaxError)",
    "source": [
      "internal/logger/logger.go:270:logger.LogIf()",
      "cmd/handler-utils.go:55:cmd.parseLocationConstraint()",
      "cmd/auth-handler.go:344:cmd.checkRequestAuthTypeCredential()",
      "cmd/auth-handler.go:296:cmd.checkRequestAuthType()",
      "cmd/bucket-handlers.go:729:cmd.objectAPIHandlers.PutBucketHandler()",
      "net/http/server.go:2047:http.HandlerFunc.ServeHTTP()"
    ]
  }
}

Which originates from the following location in minio: https://github.com/minio/minio/blob/RELEASE.2022-07-08T00-05-23Z/cmd/auth-handler.go#L344

Any idea where this malformed XML is coming from?

[basvandijk]

PR https://github.com/brendanhay/amazonka/pull/797 adds a test case for the above issue.

[endgame]

The S3 API docs for PutObject show that this endpoint should accept binary data. I see that you've uploaded some dumps to the your PR; can you please test that the AWS command-line tools and a MinIO client can successfully PutObject to your particular MinIO server?

https://github.com/minio/minio/issues/8050 may be related; I've never used MinIO.

[endgame]

Saw your remarks in #797. Is this issue a duplicate of #760?

[basvandijk]

Thanks for pointing me to that issue! It's indeed a duplicate.

I agree with your ideas how to fix this. We either need something in Env or Service that can control wether s3vhost runs or not. It could be as simp has a Bool which can toggle the behaviour.

[basvandijk]

Closing as a duplicate of https://github.com/brendanhay/amazonka/issues/760.