feat: S3Compatible storage adapter

haymaker commented 1 year ago

Description

The following PR creates a generic S3-Compatible storage adapter that extends the Utopia\Storage\Device\S3 class in order to provide for customized endpoints in vhost-style or path-style storage access.

Addressed Issue

This PR addresses #28 and attempts to do so using a least-intrusive method for code change. Scope was limited, as much as possible, to the implementation of the S3Compatible adapter. Other additions, modifications, or refactors that I may be able to address will have their own separate issues/PRs opened.

Change Details

Attempts were made to limit excess code modification. Several considerations had to be taken into account for this change:

Endpoint parameter should be configured as https://end.point without bucket suffix regardless of vhost/path styles. The S3Compatible class only supports https endpoints at the moment. This is a constraint related to the S3::call() method that would require more significant refactoring.
Vhost parameter is a simple boolean: true for vhost-style, false for path-style. Parameter brought down to S3 class and defaults to true in order to facilitate vhost vs. path styles for S3Compatible.
S3::listObjects() and S3::deletePath required modification to support vhost vs. path - there was no workaround for this.
tests\S3Base::testDeletePath() modified to support vhost vs. path.
Limitations due to current code (e.g. URI generation in S3::call() method) would need to be further addressed by other PR(s).

Tests

Unit tests available in Utopia\Tests\Storage\Device\S3CompatibleTest.
I tested against systems I have access to: MinIO, DigitalOcean Spaces, and Backblaze B2.
Tested path-style against MinIO, vhost-style against the others.
For S3CompatibleTest specifically, all tests returned OK (21 tests, 49 assertions).

haymaker commented 1 year ago

Thanks, will get these changes completed (and ensure latest main is merged into my branch).

This was my IDE - it's set to PSR-12 default for php8, I should have noticed this in the first place. I appreciate the heads up!

haymaker commented 1 year ago

Changes have been made, branch also rebased to latest upstream/main.

JoshiJoshiJoshi commented 1 year ago

Any news?

AndrewBucklin commented 12 months ago

What’s the latest on this? Really hoping to use other S3-compatible storage platforms! 😊

Thank you!

haymaker commented 12 months ago

@AndrewBucklin I'd forgotten this was still open, I thought it had been merged!

If this ends up needing to be rebased against main i can do it - it's been a while since I created this.

Depending on direction/use case, it could be used as a base for other S3-compatible storage adapter decorators.

MBDOFF commented 11 months ago

any news? @christyjacob4 @stnguyen90

docimin commented 10 months ago

Hey @haymaker Mind resolving the conflicts for us? Thanks!

haymaker commented 10 months ago

Hey @haymaker Mind resolving the conflicts for us? Thanks!

Sure @docimin, I can get to it later today.

stnguyen90 commented 6 months ago

Sorry, we're going to opt for this instead: https://github.com/utopia-php/storage/pull/103

utopia-php / storage