Minio

[ragingdave]

• Sail Version: latest
• Laravel Version: 9.21
• PHP Version: 8.0
• Host operating system: MacOS

Description:

Using Minio in conjunction with pre-signed urls seem to always generate a signature mismatch error when attempting to access a private file using a temporary Url:

<Error>
  <Code>SignatureDoesNotMatch</Code>
  <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
  <Key>test2.csv</Key>
  <BucketName>local</BucketName>
  <Resource>/local/test2.csv</Resource>
  <RequestId>17052B043977D304</RequestId>
  <HostId>9ae99075-a677-4149-b697-0cf47aa78a26</HostId>
</Error>

Steps To Reproduce:

Save a file as private access, try and generate a temporary url following the configuration of the sail docs, click the generated link, see it's broken.

Storage::disk('s3')->temporaryUrl('exports/3vyoMGv5eW/test3.csv', now()->addMinutes(60))

[driesvints]

Heya, thanks for reporting.

We'll need more info and/or code to debug this further. Can you please create a repository with the command below, commit the code that reproduces the issue as separate commits on the main/master branch and share the repository here? Please make sure that you have the latest version of the Laravel installer in order to run this command. Please also make sure you have both Git & the GitHub CLI tool properly set up.

laravel new bug-report --github="--public"

Please do not amend and create a separate commit with your custom changes. After you've posted the repository, we'll try to reproduce the issue.

Thanks!

[driesvints]

I'll reopen once a repo is provided.

[ragingdave]

Created a bug-report repo here: https://github.com/ragingdave/bug-report noted in the commit that there's some setup but it's just:

1. Actually create the bucket with private access
2. Update env with settings as outlined here: https://laravel.com/docs/9.x/sail#file-storage
3. Hit the page.

Bonus bug (maybe?) I found was that without a bucket, trying to create a file on minio just silently fails....no error no exception...nothing.

[Random72IsTaken]

@driesvints I think there's info enough to remove the label, no? 😅

[driesvints]

@ragingdave right now, when running that I get:

The GetObject operation requires non-empty parameter: Bucket

Env variables:

FILESYSTEM_DISK=s3
AWS_ACCESS_KEY_ID=sail
AWS_SECRET_ACCESS_KEY=password
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET=local
AWS_ENDPOINT=http://minio:9000
AWS_USE_PATH_STYLE_ENDPOINT=true
AWS_URL=http://localhost:9000/local

Please let me know what I need to adjust to recreate that issue.

[ragingdave]

I'm not really sure what would be the problem there as all except the .env was committed. That being said, here is the entire .env file:

APP_NAME=Laravel
APP_ENV=local
APP_KEY=base64:mZSfNEq4PNicjU8xMqNNUzxa2AFXVMUKkDvrb3DbPbM=
APP_DEBUG=true
APP_URL=http://bug-report.test

LOG_CHANNEL=stack
LOG_DEPRECATIONS_CHANNEL=null
LOG_LEVEL=debug

DB_CONNECTION=mysql
DB_HOST=mariadb
DB_PORT=3306
DB_DATABASE=bug_report
DB_USERNAME=sail
DB_PASSWORD=password

BROADCAST_DRIVER=log
CACHE_DRIVER=file
FILESYSTEM_DISK=local
QUEUE_CONNECTION=sync
SESSION_DRIVER=file
SESSION_LIFETIME=120

MEMCACHED_HOST=memcached

REDIS_HOST=redis
REDIS_PASSWORD=null
REDIS_PORT=6379

MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="${APP_NAME}"

FILESYSTEM_DISK=s3
AWS_ACCESS_KEY_ID=sail
AWS_SECRET_ACCESS_KEY=password
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET=local
AWS_ENDPOINT=http://minio:9000
AWS_USE_PATH_STYLE_ENDPOINT=true
AWS_URL=http://localhost:9000/local

PUSHER_APP_ID=
PUSHER_APP_KEY=
PUSHER_APP_SECRET=
PUSHER_HOST=
PUSHER_PORT=443
PUSHER_SCHEME=https
PUSHER_APP_CLUSTER=mt1

VITE_PUSHER_APP_KEY="${PUSHER_APP_KEY}"
VITE_PUSHER_HOST="${PUSHER_HOST}"
VITE_PUSHER_PORT="${PUSHER_PORT}"
VITE_PUSHER_SCHEME="${PUSHER_SCHEME}"
VITE_PUSHER_APP_CLUSTER="${PUSHER_APP_CLUSTER}"

Maybe something is cached config wise without that configuration somehow? I only guess that because the default .env copied from .env.example, is in that state and has an empty AWS_BUCKET env variable.

[driesvints]

The url does work (sort of). When I ssh into the container and run a curl request on the generated temporary url I do get the contents of the file:

The problem is that we need to generate an url that's forwarded. The closest I get is when I add the following to the s3 disk config:

'temporary_url' => 'http://localhost:9000',

Then I get:

I think we somehow need to tell the container to translate any "http://localhost:9000" urls into "http://minio:9000" urls to get the content.

[taylorotwell]

Can confirm the bug but no idea how to fix it. Would need to be fixed in temporaryUrl method of AwsS3V3Adapter. The temporary URL hash is generated using the minio:9000 host when we really need it generated using the localhost:9000 host. But no idea how to make that happen.

[Random72IsTaken]

But, hold on. This bit was documented to do the opposite!

Links were generated wrong for the final stored files and then were corrected...

So if AWS_URL is read for the destination files, couldn't it be read for temporary URLs as well; wherever that is? 🤔

Update: It's right underneath it! lol

[taylorotwell]

Normal Storage::url works fine when using AWS_URL. But temporary URLs are broken because the signature is generated using the minio host instead of the localhost:9000 host. I don't know if it's even possible to make the AWS SDK sign with a different host.

[driesvints]

Unfortunately it seems this is not possible with MinIO. I've sent in a PR to the docs to document this limitation: https://github.com/laravel/docs/pull/8107

[ragingdave]

I don't mean to re-open this necessarily, but doesn't this sort of hamstring a sail environment? If minio isn't capable of supporting the features that laravel/flysystem/aws-s3 provide, then wouldn't another solution be to actually fix this issue, instead of just saying "oh it doesn't work like that....sorry".

Maybe something like https://docs.localstack.cloud/aws/s3/ or https://www.zenko.io/cloudserver/ could be an actual solution? Granted I haven't vetted localstack (although it is pretty popular) but in an internal staging/qa setup, zenko's cloudserver does work and works with both url and temporary url usage from a dockerized setup.

[driesvints]

Right now, we aren't consider those. It's probably best taken up by MinIO to see if they want to offer support for this.

[joaopalopes24]

@driesvints Good evening, how are you? Do you know if there is any update on this?