search

robinrodricks / FluentStorage

A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.
MIT License
263 stars 33 forks source link

Add support for signed URLs in GCP #22

Open idofl opened 1 year ago

idofl commented 1 year ago

Both the AWS and Azure interfaces for blobs, IAwsS3BlobStorage and IAzureBlobStorage have a definition for pre-signed/SAS URLs (GetUploadUrlAsync for AWS and GetBlobSasAsync for AWS).

Can we please:

  1. Have similar support for signed-urls in GCP blobs (https://cloud.google.com/storage/docs/access-control/signed-urls)
  2. Standardize pre-signed URLs so we can have a similar API for the three common cloud providers (AWS, Azure, and GCP)
robinrodricks commented 1 year ago

Can you suggest which APIs need to change/new APIs needed so I can understand the interface you are looking at?

idofl commented 1 year ago

AWS's blob interface has definitions for pre-signed URLs for upload and download. https://github.com/robinrodricks/FluentStorage/blob/develop/FluentStorage.AWS/Blobs/IAwsS3BlobStorage.cs

Google Cloud also supports pre-signed URLs for upload and download in a similar manner of returning a signed URL: https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers#storage-signed-url-object-csharp

Aside for the mime type in the AWS method declaration, the same method declaration can be used for GCP.

From what I can tell, the GCP implementation does not have a base interface, but rather inherit from GenericBlobStorage. https://github.com/robinrodricks/FluentStorage/blob/develop/FluentStorage.GCP/Blobs/GoogleCloudStorageBlobStorage.cs

If I had to make a suggestion, I would probably add an interface to extend the capabilities of the GenericBlobStorage for Google Cloud as a first step, and eventually update the extended IBlobStorage interface to support pre-signed URLs, as they exist in most cloud storage providers. https://github.com/robinrodricks/FluentStorage/blob/develop/FluentStorage/Blobs/IBlobStorage.cs

The implementation for GCP can either sign locally if a service account key is available. If a signing key is not available, the implementation can call the signBlob API to sign the URL. https://cloud.google.com/storage/docs/access-control/signed-urls#signing-iam