search

snipe / snipe-it

A free open source IT asset/license management system
https://snipeitapp.com
GNU Affero General Public License v3.0
11.21k stars 3.2k forks source link

S3 Public Upload is throwing Access Denied error #10077

Closed nubpro closed 3 years ago

nubpro commented 3 years ago

Debug mode

Describe the bug

I'm running a docker image, and therefore, I'm using S3 bucket as the storage layer. I've configured both public and private bucket through the ENV file. I've parked two individual S3 buckets under different AWS user, and have allocated the correct permissions respectively. I've verified this by using Postman to upload files to my S3 bucket with the credentials I've set up in the previous step.

The problem occurs when I'm trying to upload an asset image, I get hit by this error "Access Denied". The same does not happen to uploading asset files which are using the private s3 bucket.

Full error:

Error executing "PutObject" on "https://prod-xx-public.s3.ap-southeast-1.amazonaws.com/assets/asset-image-PD5DMxFcEt.png"; AWS HTTP error: Client error: `PUT https://prod-xx-public.s3.ap-southeast-1.amazonaws.com/assets/asset-image-PD5DMxFcEt.png` resulted in a `403 Forbidden` response: <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>HS765J (truncated...) AccessDenied (client): Access Denied - <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>HS765JPZ3TY</RequestId><HostId>7IqqpZdyxpVkK7KK4VjoROUFk+BhwKCfpO8YS/eebBBmjxz

Docker ENV file:

PRIVATE_AWS_ACCESS_KEY_ID="xx"
PRIVATE_AWS_BUCKET="prod-xx-private"
PRIVATE_AWS_BUCKET_ROOT="null"
PRIVATE_AWS_DEFAULT_REGION="ap-southeast-1"
PRIVATE_AWS_SECRET_ACCESS_KEY="xx"
PRIVATE_AWS_URL="https://prod-xx-private.s3.ap-southeast-1.amazonaws.com"
PRIVATE_FILESYSTEM_DISK="s3_private"

PUBLIC_AWS_ACCESS_KEY_ID="xx"
PUBLIC_AWS_BUCKET="prod-xx-public"
PUBLIC_AWS_BUCKET_ROOT="null"
PUBLIC_AWS_DEFAULT_REGION="ap-southeast-1"
PUBLIC_AWS_SECRET_ACCESS_KEY="xx"
PUBLIC_AWS_URL="https://prod-xx-public.s3.ap-southeast-1.amazonaws.com"
PUBLIC_FILESYSTEM_DISK="s3_public"

Reproduction steps

  1. Setup docker
  2. Setup ENV files to use S3 buckets for storage
  3. Try uploading a asset image (uses public s3 bucket)
  4. Fail to upload. Access denied.

Expected behavior

Able to upload asset image

Screenshots

No response

Snipe-IT Version

5.2.0

Operating System

Ubuntu

Web Server

Nginx

PHP Version

7 I think, Im using docker

Operating System

Window

Browser

Chrome

Version

92

Device

No response

Operating System

No response

Browser

No response

Version

No response

Error messages

No response

Additional context

No response

welcome[bot] commented 3 years ago

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. We get a lot of issues on this repo, so please be patient and we will get back to you as soon as we can.

snipe commented 3 years ago

For PUBLIC_AWS_BUCKET_ROOT, do you have null in quotes, as shown in your example env, or just null?

nubpro commented 3 years ago

For PUBLIC_AWS_BUCKET_ROOT, do you have null in quotes, as shown in your example env, or just null?

They are in quotes indeed.

snipe commented 3 years ago

It will try to use the literal string "null" if it's in quotes.

nubpro commented 3 years ago

It will try to use the literal string "null" if it's in quotes.

It works for the private s3 bucket where the null string is omitted from the actual path. The same applies to the public bucket when I tried it.

snipe commented 3 years ago

It should still not be in quotes unless that's actually part of the bucket URL

nubpro commented 3 years ago

It should still not be in quotes unless that's actually part of the bucket URL

I understand, I will change this. I dont thk it is causing the bug? I also tested with the null string replaced with something proper, the same problem still occurs

snipe commented 3 years ago

I'm unable to reproduce this on the demo. https://demo.snipeitapp.com/hardware/1375

(Uploads are disabled on the demo though - I temporarily disabled them to test this. I'll leave it open for now so you can test for yourself tho.)

Snipe-IT Demo
nubpro commented 3 years ago

I'm unable to reproduce this on the demo. https://demo.snipeitapp.com/hardware/1375

(Uploads are disabled on the demo though - I temporarily disabled them to test this. I'll leave it open for now so you can test for yourself tho.)

Snipe-IT Demo

Do u mind to leave them enabled til tomorrow? Im away currently 😅

Is it possible that u can share your masked ENV files so I can try to match it up?

Snipe-IT Demo
snipe commented 3 years ago 
# --------------------------------------------
# REQUIRED: UPLOADED FILE STORAGE SETTINGS
# --------------------------------------------
PRIVATE_FILESYSTEM_DISK=s3_private
PUBLIC_FILESYSTEM_DISK=s3_public

FILESYSTEM_DISK=s3

# --------------------------------------------
# PUBLIC S3 Settings
# --------------------------------------------
PUBLIC_AWS_SECRET_ACCESS_KEY='XXXXXXXXXXXX'
PUBLIC_AWS_ACCESS_KEY_ID='YYYYYYYYY'
PUBLIC_AWS_DEFAULT_REGION='us-west-2'
PUBLIC_AWS_BUCKET=snipe-flysystem-public-test
PUBLIC_AWS_URL='https://XXXX.s3-us-west-2.amazonaws.com/'

# --------------------------------------------
# PRIVATE S3 Settings
# --------------------------------------------
PRIVATE_AWS_SECRET_ACCESS_KEY='XXXXXXXXXXXX'
PRIVATE_AWS_ACCESS_KEY_ID='YYYYYYYYYY'
PRIVATE_AWS_DEFAULT_REGION='us-west-2'
PRIVATE_AWS_BUCKET=snipe-flysystem-private-test
nubpro commented 3 years ago

--------------------------------------------

REQUIRED: UPLOADED FILE STORAGE SETTINGS

--------------------------------------------

PRIVATE_FILESYSTEM_DISK=s3_private PUBLIC_FILESYSTEM_DISK=s3_public

FILESYSTEM_DISK=s3

--------------------------------------------

PUBLIC S3 Settings

--------------------------------------------

PUBLIC_AWS_SECRET_ACCESS_KEY='XXXXXXXXXXXX' PUBLIC_AWS_ACCESS_KEY_ID='YYYYYYYYY' PUBLIC_AWS_DEFAULT_REGION='us-west-2' PUBLIC_AWS_BUCKET=snipe-flysystem-public-test PUBLIC_AWS_URL='https://XXXX.s3-us-west-2.amazonaws.com/'

--------------------------------------------

PRIVATE S3 Settings

--------------------------------------------

PRIVATE_AWS_SECRET_ACCESS_KEY='XXXXXXXXXXXX' PRIVATE_AWS_ACCESS_KEY_ID='YYYYYYYYYY' PRIVATE_AWS_DEFAULT_REGION='us-west-2' PRIVATE_AWS_BUCKET=snipe-flysystem-private-test

Interesting I dont thk I have set FILE_SYSTEM variable as s3. Lemme get back to u when Im home

nubpro commented 3 years ago

Some good news! Turns out I misconfigured the IAM permissions, it was severely lacking of some essentially policies after referring to https://flysystem.thephpleague.com/v1/docs/adapter/aws-s3-v3/

Below is the required policies mandated by the flysystem:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1420044805001",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:ReplicateObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your-bucket-name",
                "arn:aws:s3:::your-bucket-name/*"
            ]
        }
    ]
}

I believe this can be better communicated over on the docs site, with that said, I'm closing this!

Aws S3 Adapter V3 - Flysystem
Filesystem abstraction library for PHP.