Upload URL Generator

[alexander-schranz]

Feature Request

Some storages are supporting to generate an URL which can be used by a client to upload the file directly to the storage. Example here is S3 https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html. I did not yet use that mechanism of S3 but thought it would an interesting feature. In azure I did only found something like azure sas url upload but not 100% sure if it is the same (sounds a lot more complicated but could maybe achievied maybe via this: https://learn.microsoft.com/en-us/azure/storage/common/storage-sas-overview). For google cloud I did found this here: https://cloud.google.com/storage/docs/samples/storage-generate-upload-signed-url-v4?hl=de#storage_generate_upload_signed_url_v4-php.

Q	A
Flysystem Version	3.x
Adapter Name	s3, gcloud, ...
Adapter version	3.x

Scenario / Use-case

Directly upload files from the Browser to the storage instead of streaming them over the PHP application server.

Summary

A upload url generator support for some adapters, could reduce load on servers as uploads are directly done to the external storage and don't need to stream through the webserver.

There was a similar issue here #1423 but maybe with implementing of UrlGenerator and TemporaryUrlGenerator you see more value for having this kind of interface, feature for adapters.

[frankdejonge]

It's an interesting idea, I'll see what I can come up with :)

[dkarlovi]

Example use case https://github.com/pimcore/pimcore/issues/8532

In azure I did only found something like azure sas url upload but not 100% sure if it is the same

It is, we used it exactly like that in several projects. S3, Azure and GCP all support basically the same feature. The idea here is to see if we can abstract not just write but also read. Namely, SAS URLs (Azure terminology) are used for authorization and you can make them read-write (scenario: "upload here") or just read (scenario: here's a 15 min permission to download this asset). It's general access control, not just upload.

Having this feature would allow implementing access control to existing assets you want to keep private:

1. I want to download an asset
2. asset is marked private, I cannot access it directly
3. I log into the system which is managing the assets (some sort of DAM system like Pimcore in this case)
4. I ask the system to download the asset
5. the system generates the SAS URL because it thinks I'm allowed to access it (because of my role or whatever)
6. I follow the URL and download the asset directly off cloud storage without further interaction with the DAM system

The more correct terminology from Flysystem's POV might be "access request" (RO, RW).

[alexander-schranz]

@dkarlovi the download part is already supported. You find it here in the documentation:

• Temporary Urls: https://flysystem.thephpleague.com/docs/usage/temporary-urls/
• Public Urls: https://flysystem.thephpleague.com/docs/usage/public-urls/

[dkarlovi]

Ah, didn't see that, nice!

These concepts seem like they're somewhat connected (at least this applies to ABS), it might make sense to see if they can be streamlined in some way maybe.

[frankdejonge]

If anybody has found how to actually use those upload URL's, I'd love to see it. I find the documentation about this very lacking. I didn't find AWS examples, nor did I see how the laravel stuff is used.

[dkarlovi]

@frankdejonge assuming you have the SAS URL, you just PUT to it. At least it works like that with Azure SDK, we created an app where the PHP SDK (with a connection string) created the URL and passed to JS SDK (without the connection string). It's the target path.

[frankdejonge]

@dkarlovi any chance you can supply a curl request in PHP that would work?

[alexander-schranz]

If I look at the AWS documentation I hope this is the correct one: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/s3-presigned-post.html#create-postobjectv4. It seems not only be simple URI where we can send the POST to it, also additional post data seems to be required. They seems to be generated here: https://github.com/aws/aws-sdk-php/blob/d5586c88e7ef943f9cc47597c53073b213b9339e/src/S3/PostObjectV4.php#L166-L193

So we maybe require a combination of URI + input data which the interface need to be returned, maybe also the HTTP Method if its PUT or POST.

The google cloud implementation looks a little bit easier: https://cloud.google.com/storage/docs/samples/storage-generate-upload-signed-url-v4?hl=de#storage_generate_upload_signed_url_v4-php

The Azure docs is really confusing here. Looking at there examples I can not find a direct upload example but maybe we need just change the SAS to allow write here:

https://github.com/Azure/azure-storage-php/blob/4cd7470e517022faa80bed98d3c7518aa512ea97/samples/FileSamples.php#L270

Sadly I current have not yet a working code for you as I would first need to create accounts for the services, the next weeks also little bit stressful, but think I could give, if not somebody else having a look at it, at begin of May some prototype.

[frankdejonge]

This is exact the research I did a while back, which seems to indicate all the providers support a very different model. Either forms or cURL requests, require get params or headers or form fields. The real problem is distiling a unified model out of this which is better than just using the clients directly. I think we can disregard the Azure path btw, since they're nope'ing PHP.

[Pink0410]

My project also has this requirement, generate the bucket's pre-signature,JavaScript calls the pre-signature, upload to the bucket, and the bucket calls back to my project. I think the best way is the public FilesystemAdapter