Open rtrzebinski-usc opened 1 year ago
I found out that the url to download the file would be http://localhost:49409/download/storage/v1/b/foo/o/bar
- but how do I check whether file was uploaded using the code (reader) so I don't receive failed to create reader, storage: object doesn't exist
error?
I'm also able to reproduce the above (with my own implementation). The issue is related to the -public-host
argument not being set.
If I adapt the go example to create an object and then read that object, this only works -public-host localhost:8080
is set (ci script). Without it, I also get storage: object doesn't exist
when executing it.
- Cmd: []string{"-scheme", "http", "-port", "4443", "-public-host", "localhost:4443"}
+ Cmd: []string{"-scheme", "http", "-port", "4443", "-public-host", "0.0.0.0"}
Exploring the source code, you can see that one of the downloadObject
's MatcherFunc
's uses the publicHostMatcher
(which uses the public-host
argument and publicHost
config.
https://github.com/fsouza/fake-gcs-server/blob/071372eda6e3cfd4db175a83ef4fa6cb75f024de/fakestorage/server.go#L274-L277
So taking a look at this MatcherFunc
, it will match requests only for a specific port if one is given, but falls back to matching any port if none is given.
https://github.com/fsouza/fake-gcs-server/blob/071372eda6e3cfd4db175a83ef4fa6cb75f024de/fakestorage/server.go#L293-L300
The documentation in the README mentions setting -public-host
but doesn't mention that
-public-host
for downloadObject
(just for pre-signed urls).-public-host
It might be nice to document this, because it also cost me about 2 hours to work through everything (especially because I initially thought it was an implementation bug in my code)?
@dezyh thanks for digging into this. Since you did the investigation, do you also want to send a PR with improvements to the docs? I can do it if you prefer!
I'll make a readme PR, just wanted to check that I wasn't missing something obvious.
@dezyh oh, that sounds good. I'll dig into that sample code to see if there's a better fix, but yeah sounds like we need to clarify the role of public-host
in the docs.
@fsouza @dezyh I followed the instructions given here and I just ran the code given above and set the public-host
to 0.0.0.0
. But I still face the same issue. Am I missing anything?
2022/11/22 12:12:41 failed to create reader, storage: object doesn't exist
sleeping..
@rtrzebinski-usc were you able to run this test case successfully?
Sorry, I was busy for a while.
I'm personally using dockertest and running containers inside a docker network. This works perfectly both local and on GitHub Actions with my above comment.
One small subtlety, is that connecting through the network's hostname for the container (not sure on terminology) doesn't work in GitHub Actions. While this is partly dockertest specific, it might also apply with testcontainers.
CloudStorageEndpoint = fmt.Sprintf("%s:%s", cloudStorageContainer.Container.NetworkSettings.Networks["test"].IPAddress, CloudStoragePort)
I have to connect through a port that's bound to the default network (again, sorry don't know the correct terminology)
CloudStorageEndpoint = fmt.Sprintf("0.0.0.0:%s", cloudStorageContainer.GetPort(fmt.Sprintf("%s/tcp", CloudStoragePort)))
Hi, here is my issue with the emulator - I use
testcontainers
for integration test, uploading works fine (I see the file in the container) but there is no way to access it via HTTP or from the code, here is the script to reproduce:When you run it will print the base url like
http://localhost:49405/storage/v1
- I try to add bucked and object name and access likehttp://localhost:49405/storage/v1/foo/bar
- getting 404.Then since script is paused by
time.Sleep(1000 * time.Second)
I can inspect the image - I see that:What I do wrong? Any help will be appreciated, thank you :)