New R/W pre-signed URL support requires the `x-ms-blob-type` header with PUT

adobe / aio-lib-files

An abstraction on top of cloud blob storage exposing a file system like API

https://www.adobe.io

Apache License 2.0

10 stars 14 forks source link

New R/W pre-signed URL support requires the `x-ms-blob-type` header with PUT #67

Closed bobvanmanen closed 3 years ago

bobvanmanen commented 4 years ago

Expected Behaviour

This should work:

const url = await files.generatePresignURL('test.png', {
     expiryInSeconds: 60,
     permissions: 'rwd'
});
await axios.put(url, { hello: 'world' });

Actual Behaviour

It fails because x-ms-blob-type: BlockBlob is a required header. See: https://docs.microsoft.com/en-us/rest/api/storageservices/put-blob

Platform and Version

This is an Azure API requirement.

bobvanmanen commented 4 years ago

I wonder if the CDN could add the x-ms-blob-type: BlockBlob header to PUT requests on behalf of the customer.

aiojbot commented 4 years ago

JIRA issue created: https://jira.corp.adobe.com/browse/ACNA-941

meryllblanchet commented 4 years ago

Thanks for the report, @bobvanmanen !

Extra thoughts from my side for design purpose:

Every API change must not break backwards-compatibility and abstracted implementation/vendor-specific details
While your suggestion to manage the header at CDN level looks good in regards with the previous point, it might be an issue if we need to provide developers the possibility to generate URLs pointing directly to the storage domain w/o CDN delivery (e.g. for namespace to namespace exchange of the payload reference)

alexkli commented 4 years ago

Note that you can add the header in to the presigned url. The azure implementation inside aio-lib-files can do this when it presigns a PUT, as it only deals with blobs here. This makes it transparent and backwards compatible.

We do the same in AEM.

sandeep-paliwal commented 4 years ago

@alexkli The Azure SAS only allows to specify few response headers, in this case we will need to support request headers, so it won't be possible to fix this in sdk. ref sas doc

Let me know if I am wrong here.

meryllblanchet commented 4 years ago

Meanwhile, we will release a config change at CDN level as discussed above in the ticket. If we confirm there's a way to make it at SDK library level, we'll implement it as a second step to automatically provide a similar support in both OOTB and Bring Your Own storages.

meryllblanchet commented 4 years ago

@bobvanmanen the fix is in place at CDN level, could you please give it a new try and confirm that your requests are executed as expected?

alexkli commented 3 years ago

@meryllblanchet are these presigned URLs always going through a CDN? For clients inside data centers (which is the case for Nui) this will be slower than direct Azure BS or AWS S3 URLs.

meryllblanchet commented 3 years ago

@alexkli for now yes. We're waiting on first concrete performance feedback (e.g. Nui workers) to work on and provide optimization features.

FYI, we have https://github.com/adobe/aio-lib-files/pull/65 in case this will be effectively needed, but we are also considering a tighter integration between the SDK and Runtime capabilities to serve the files (with or without CDN) in a more seamless manner to the end-users/systems.