gaul / s3proxy

Access other storage backends via the S3 API
Apache License 2.0
1.75k stars 228 forks source link

If using configuration as written on the wiki Azure Blob does not work (empty endpoint url). #469

Open EraYaN opened 1 year ago

EraYaN commented 1 year ago

So the problem is if you leave the endpoint empty it is not correctly inferred. Leading to BadRequest error for Puts and empty errors for MultiPart uploads. When you then put the full endpoint URL in there it does work.

Used config is:

s3proxy.endpoint=http://127.0.0.1:8080
s3proxy.authorization=aws-v2-or-v4
s3proxy.identity=local-identity
s3proxy.credential=local-credential
jclouds.azureblob.auth=azureKey
jclouds.provider=azureblob
jclouds.identity=xxxxxxxxx
jclouds.credential=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

But the jcould.endpoint=https://xxxxxxxxx.blob.core.windows.net entry is mandatory apparently.

If on startup the endpoint is empty maybe emitting an error would be helpful. Or returning some error to the client instead of just bad request.

timuralp commented 1 year ago

I know there is an issue if jclouds.endpoint is set to an empty string, but this configuration succeeds for me:

jclouds.provider=azureblob
jclouds.azureblob_auth=azureKey
jclouds.identity=<id>
jclouds.credential=<key>
s3proxy.authorization=aws-v4
s3proxy.identity=foo
s3proxy.credential=bar
s3proxy.endpoint=http://localhost:10080

I tested PUT of a small object. Can you share more details about the failure?

EraYaN commented 1 year ago

It's been a while and the logs are long lost. The main thing I remember is that aws CLI did not want to upload essentially anything with the endpoint being empty.

alifirat commented 1 year ago

To me the jclouds.endpoint cannot be empty. If you enable the trace logs you can see that the jclouds endpoint is used to do the PUT request on the Azure blob storage.