Configurable endpoint

[blogle]

Currently much of the uris are built from hard-coded associated constants in the Client struct. it would be great if the Client::Endpoint was configurable (or overridable by STORAGE_EMULATOR_HOST) so users could program against a test server as demonstrated in this project https://github.com/fsouza/fake-gcs-server/tree/main/examples

it would appear that others are interested in this functionality (see: https://github.com/pingcap/br/issues/258) and they have investigated crudely swapping out the uri from tame_gcs in the tikv gcs backend

[Hirevo]

Hello !

Sorry for my long absence.
This seems straightforward to implement, if it suffices to add a constructor to allow passing a custom endpoint to the Client types.
One question though is does this require to implement authentication differently than for normal GCP ?
From what I can see, the particular emulator works over plain-text HTTP and without authentication, if I understood correctly ?
If so, we would need a way to kinda 'disable' authentication as well, I guess.

[Roba1993]

@blogle I had the same requirement as you. I looked into this topic and decided for myself to go down a different route. Instead of usinf the fake-gcs-server I created the following library https://github.com/Roba1993/stow/actions With this library you can store the files locally or on the GCP based upon the inital config.

[blogle]

Hello !

Sorry for my long absence. This seems straightforward to implement, if it suffices to add a constructor to allow passing a custom endpoint to the Client types. One question though is does this require to implement authentication differently than for normal GCP ? From what I can see, the particular emulator works over plain-text HTTP and without authentication, if I understood correctly ? If so, we would need a way to kinda 'disable' authentication as well, I guess.

Yeah looking at the different client examples on the fake-gcs-server repo, they either configure the client endpoint (see https://github.com/fsouza/fake-gcs-server/blob/main/examples/go/main.go#L27) or instantiate the client using AnonymousCredentials. As you mention I believe auth would likely need to be disabled when hitting the local/test server.

[Tarkin25]

@Hirevo

Hello !

Sorry for my long absence. This seems straightforward to implement, if it suffices to add a constructor to allow passing a custom endpoint to the Client types. One question though is does this require to implement authentication differently than for normal GCP ? From what I can see, the particular emulator works over plain-text HTTP and without authentication, if I understood correctly ? If so, we would need a way to kinda 'disable' authentication as well, I guess.

I have looked at the code of the cloud-pubsub crate, and from what I saw, there is no need to disable authentication. I did only test it for the pubsub emulator, not for any of the other ones.

Simply configuring the endpoint URL to the local emulator endpoint might just work as seen here: https://github.com/kayleg/cloud-pubsub/blob/master/src/subscription.rs