search

minio / minio-js

MinIO Client SDK for Javascript
https://docs.min.io/docs/javascript-client-quickstart-guide.html
Apache License 2.0
932 stars 274 forks source link

ambiguous S3 API Reference `GetBucketLocation` causes incompatiblity with SeaweedFS S3 implementation #1213

Closed robertgroh closed 11 months ago

robertgroh commented 11 months ago

Hi, I am using minio-js to access a Minio S3 storage in a React web app, which works fine. A requirement of the project is to also support the S3 implementation of SeaweedFS and there I have problems and getting the following error, when I try to list the content inside a bucket

Uncaught (in promise) TypeError: region should be of type "string"
    signV4 signing.ts:218
    _callee5$ client.ts:613
    ...
    makeRequestStreamAsync client.ts:637
    _callee3$ client.ts:535
    ...
    makeRequestAsync client.ts:537
    makeRequest client.ts:723
    listObjectsV2Query minio.js:971
    _read minio.js:1036
   ...

I tracked the issue down to the different responses which Minio and SeaweedFS gives for the GetBucketLocation query:

which according to the Amazon Simple Storage Service (S3) API Reference of GetBucketLocation seems to be the correct Response Syntax:

HTTP/1.1 200 <?xml version="1.0" encoding="UTF-8"?> <LocationConstraint> <LocationConstraint>string</LocationConstraint> </LocationConstraint>

But unfortunately the S3 API Reference also give the unnested LocationConstraint as a "Sample Response":

<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">us-west-2</LocationConstraint>

Can minio-js support both Response Syntaxes?

A possible unit test for src/internal/xml-parser.ts#parseBucketRegion(xml: string) might be

import { parseBucketRegion } from '../../src/internal/xml-parser.ts'

describe('xml-parser', () => {
  describe('#parseBucketRegion()', () => {
    describe('of a nested LocationConstraint', () => {
      const xml = `
        <?xml version="1.0" encoding="UTF-8"?>
        <LocationConstraint>
          <LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></LocationConstraint>
        </LocationConstraint>
        `

      it('should get the inner LocationConstraint', () => {
        const region = parseBucketRegion(xml)
        assert.equal(region, '')
      })
    })

    describe('of an unnested LocationConstraint', () => {
     const xml = `
        <?xml version="1.0" encoding="UTF-8"?>
        <LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/"></LocationConstraint>
        `
      it('should get the LocationConstraint value', () => {
        const  region = parseBucketRegion(xml)
        assert.equal(region, '')
      })
    })
  })
})
prakashsvmx commented 11 months ago

The implementation of MinIO server is S3 compatible. And all SDKs adhere to the specification.

We will not be able to implement/support arbitrary non compliant S3 providers.

robertgroh commented 11 months ago

@prakashsvmx Thank you for the fast reply. Would you be so kind to point me to the correct S3 specification?

prakashsvmx commented 11 months ago

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html

S3 Response as per spec:

         <?xml version="1.0" encoding="UTF-8"?>
         <LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">us-west-2</LocationConstraint>

MinIO Response: ( with mc admin trace <ALIAS> -v )

<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">us-east-1</LocationConstraint>

complete trace

play.min.io [REQUEST s3.GetBucketLocation] [2023-10-17T20:06:33.109] [Client IP: 106.197.40.218]
play.min.io GET /test-bucket/?location=
play.min.io Proto: HTTP/1.1
play.min.io Host: play.min.io
play.min.io X-Forwarded-Proto: https
play.min.io X-Real-Ip: 106.197.40.218
play.min.io Content-Length: 0
play.min.io User-Agent: MinIO (linux; amd64) minio-go/v7.0.63 mc/RELEASE.2023-09-29T16-41-22Z
play.min.io X-Amz-Content-Sha256: UNSIGNED-PAYLOAD
play.min.io X-Amz-Date: 20231017T143632Z
play.min.io X-Forwarded-For: 106.197.40.218
play.min.io Authorization: AWS4-HMAC-SHA256 Credential=Q3AM3UQ867SPQQA43P2F/20231017/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=582cfdbe3341318c83d6dcf41e3cc18f719f7168156b3d45d97fe9a784deede1
play.min.io X-Nginx-Proxy: true
play.min.io 
play.min.io [RESPONSE] [2023-10-17T20:06:33.109] [ Duration 244µs TTFB 202.771µs ↑ 135 B  ↓ 137 B ]
play.min.io 200 OK
play.min.io Accept-Ranges: bytes
play.min.io Content-Length: 137
play.min.io Server: MinIO
play.min.io Vary: Origin,Accept-Encoding
play.min.io X-Amz-Bucket-Region: us-east-1
play.min.io X-Amz-Request-Id: 178EEBD12790BAC9
play.min.io Content-Type: application/xml
play.min.io Strict-Transport-Security: max-age=31536000; includeSubDomains
play.min.io X-Amz-Id-2: 3e996b2f640d7e065d3a5c4e39a5538cefb82e3e77771990265e4698d8681eac
play.min.io X-Content-Type-Options: nosniff
play.min.io X-Xss-Protection: 1; mode=block
play.min.io <?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint xmlns="http://s3.amazonaws.com/doc/2006-03-01/">us-east-1</LocationConstraint>
play.min.io
robertgroh commented 11 months ago

Thanks again, unfortunately the same spec also has this:

Response Syntax

HTTP/1.1 200
<?xml version="1.0" encoding="UTF-8"?>
<LocationConstraint>
   <LocationConstraint>string</LocationConstraint>
</LocationConstraint>

see: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html#API_GetBucketLocation_ResponseSyntax

I used the Feedback form there, to ask to clarify the difference between Response Syntax and Sample Response, lets see when someone at Amazon comes around to it.