search

amol- / depot

Toolkit for storing files and attachments in web applications
MIT License
161 stars 41 forks source link

Added support for Google Cloud Storage #80

Closed maz1987in closed 1 year ago

maz1987in commented 1 year ago

Added support for Google Cloud Storage

maz1987in commented 1 year ago

@amol- Could you please review the code?

amol- commented 1 year ago

@amol- Could you please review the code?

I took a quick look already, but I was thinking on how to evolve the CI in a robust way to test GCS support. It seems there are multiple local implementations of GCS, but it's unclear what the best one would be, they all seem to be a bit unstable. See https://github.com/googleapis/google-cloud-python/issues/4897

We could in theory use Minio, but Minio provide S3 APIv2, which is compatible with GCS, is not natively GCS.

maz1987in commented 1 year ago

@amol- I have added all required but not CI as I am new to it and I tried but had no luck until now

maz1987in commented 1 year ago

@amol- I tried to add Minio as an emulator but concluded that the Minio is incompatible with GCP storage.

amol- commented 1 year ago

@amol- I tried to add Minio as an emulator but concluded that the Minio is incompatible with GCP storage.

Thanks for trying. I guess I'll have to setup a real GCP storage node for CI.

maz1987in commented 1 year ago

@amol- I have added emulators for GCS Please check and marge

maz1987in commented 1 year ago

@amol- I has able to find a good emulator for GCS and I tested it without an issue. please check it.

amol- commented 1 year ago

GCS support is now on master, I had to do some tweaks to make it fully compliant to the depot api and make all tests pass, and it seems meanwhile you did changes too. Could you please rebase your PR on top of master if there are further changes you wanted to apply?

I noticed that while you were setting up the emulation, I was setting up testing against the real API. I think it would be great to keep both. Even if we don't use it, there is still a value in allowing DEPOT users to run their apps against the emulator instead of the real API.

PS: The tests are currently failing, but that's related to a change in S3, the GCS tests are correctly passing.

maz1987in commented 1 year ago

@amol- Sorry I have done will all the changes.

maz1987in commented 1 year ago

@amol- Sorry for the inconvenience find my last commit to fix the merge