S3 403 Forbidden - perhaps check your credentials?

[grachevko]

Describe the bug S3 node call S3 endpoint to get region of bucket before request to any method except BucketList/BucketCreate/BucketDelete. In case where credentials doesn't have access to GetBucketLocation operation it cause error 403 check credentials. This is very confusing and cost me several hours to debug because documentation doesn't contain any information about this requirements.

I guess this question was about the same https://community.n8n.io/t/getting-403-forbidden-while-downloading-a-file-from-s3-even-though-the-credentials-are-correct/6437

To Reproduce

1. Create credentials without permission on GetBucketLocation
2. Create S3 node with resource File and Get Many operation
3. See error

Expected behavior

1. Documention contain correct requirements about BucketGetLocation permission.
2. Ignore error to get Bucket Region as it done in minio-go client https://github.com/minio/minio-go/blob/master/bucket-cache.go#L120 They just use default region if any error caused on fetching Bucket Region.

Moreover they also use region from configuration if it present https://github.com/minio/minio-go/blob/master/bucket-cache.go#L91 Our credentials also has region field, maybe we should use it without pre request if it present.

Additional context We use OBS as a S3 server. Our version of this server return empty response on Bucket Location method, which cause an ERROR: Cannot read properties of undefined (reading '_') on this line https://github.com/n8n-io/n8n/blob/n8n%400.223.0/packages/nodes-base/nodes/S3/S3.node.ts#L700

Ofcouse this related to OBS more then n8n, but for now no workaround to avoid this. No option to turn off prefetching location or use location from credentials/options.

[Joffcom]

Hey @grachevko

In the credential if you check the "force path style" option does it then work for you?

[grachevko]

Hey @Joffcom

How it can be related to described issue? OBS support both path and domain style, checking or unchecking this does make nothing for me.

[Joffcom]

Hey @grachevko

It was my first thought as it is what is needed when connecting to Minio.

I will see if I can get OBS set up when I am back next Tuesday to see if we need to do anything special.

[grachevko]

You can create custom policy in minio without allow access to GetBucketLocation

Default readonly policy in minio have this

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

Empty response from OBS doesn't matter. Matter is S3 node fetch location on every operation even if region already preset in credentials. minio client doesn't fetch it if it present in options.

[Joffcom]

@grachevko ah I get you I missed the permission bit. Will need to think about whether we treat this as a bug or a feature request but it looks like something we should sort out soonish.

[fraenkDUS]

Same problem here with ERROR: Cannot read properties of undefined (reading '_') when using Get Many on File or Folder from a S3 compatible Object Storage Service (OBS).

So the only workarounds so far would be to modify the S3 node or make HTTP requests, right?

[grachevko]

@fraenkDUS assign s3:GetBucketLocation permission for your role

[Joffcom]

I really should do something with this, I will move it up my list.

[fraenkDUS]

@fraenkDUS assign s3:GetBucketLocation permission for your role

Thanks, but it doesn't change anything.

I think you already mentioned the root cause: For every request (Search, Get All etc.) it sends and expects the region but that's not supported by S3. For example: https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

I got at least the Bucket > Search working by removing all references to "region" in the S3 GenericFuntions.js and S3.node.js.

File & Folder > Get All doesn't return anything yet, might be another issue.