Open ckortekaas opened 2 years ago
Thank you for reporting. I will have a look at it tomorrow :)
I typically use the environment variable alternative for the flags were possible when running the docker image. I didn't see the EOF file before. I will try to reproduce the issue locally.
@ckortekaas I typically don't use AWS myself, do you happen to know if Minio would be good enough or would I need a Amazon S3?
Hi @weyert, this EOF happens even just using the local version, eg sh'ing into the container and running:
./tapico-turborepo-remote-cache --turbo-token=testtoken --kind=local -v
It shows:
# ./tapico-turborepo-remote-cache --turbo-token=testtoken --kind=local -v
projectID: kind: local localStoragePath: aws.endpoint: google.endpoint: google.credentialsJsonPath: Starting the Tapico Turborepo remote cache server
But EOF errors still show up when trying to connect. So something fundamental about container networking from a MacOS host and maybe something with the Go network listener?
@weyert ok I've figured it out, this works running inside the container via a shell:
LISTEN_ADDRESS=:8080 ./tapico-turborepo-remote-cache --turbo-token=testtoken --kind=local -v
That tells http.ListenAndServe to listen on any hostname, whereas at the moment the default is to only listen on 'localhost:8080'. Since calling a docker container from another container or from the host will usually make the call via a different hostname the Go library will never accept this I suspect.
So I guess the fix for this is maybe adding to the LISTEN_ADDRESS as a configuration flag, or defaulting to listening to just ':8080' (any host) instead of 'localhost:8080' on line 547 of main.go?:
err := http.ListenAndServe("localhost:8080", loggedRouter)
Yeah, probably we should just listen to 0.0.0.0:8080
instead. I should update the README file with the latest state of things.
My apologies for the lack of documenting this LISTEN_ADDRESS
.
Awesome! BUCKET_NAME is another good env var to add to the docs too. AWS ECS Fargate as well as docker-compose were way easier to get the syntax right using environment variables. Thanks again for your inspiring project!
Awesome, glad you like the project :)
Don't hesitate to raise a PR for the docs or instructions how to get it running on AWS. That would be amazing!
@ckortekaas I have created an example and updated the README.md
in the root. Would you be so kind and have a look at the two README files and the docker-compose
example and whether you think it would be a useful example?
See: https://github.com/Tapico/tapico-turborepo-remote-cache/tree/main/examples/with-docker-compose
I also tried to reproduce the EOF issue of this ticket but I haven't been able to make it happen using the above mentioned docker-compose file. Could it be a AWS S3 specific thing? 🤔
@weyert That looks great! I suspect the reason you might not have the issue with EOF is a difference in host OS. I am using macOS 12.1 with Docker Desktop 4.3.2 (the latest version). Are you perhaps on a different host OS and older or different version of Docker Desktop?
Ah I am on 4.3.1 and macOS 11. I am updating Docker now and will try again. Are you seeing the issue with the docker compose example?
I haven't had much luck yet to reproduce your problem. I will try to upgrade to macOS 12.1
@ckortekaas I have seen it occasionally earlier this week. I am wondering if it could be to do with some default maximum upload size in Golang for the PUT route? I will look into this
I am wondering if this ticket is related? https://github.com/vercel/turborepo/issues/487 Could the problem be on Turbo's side?
Firstly - thanks for this extremely useful addon for Turborepo! Very impressive!
I have no problems using the downloadable executable, but using the docker container from the registry or built locally gives this error when turbo tries to connect eg:
That's running with an apiUrl and teamId set in
./turbo/config.json
via turbo eg:Other things I've tried result in a connection refused error instead, so EOF seems to imply it's connecting but failing to PUT. I'm using S3 with valid credentials that work with the non-docker version. I've even modified the container to execute inside it manually various commands to make sure it's not that the command arguments are not being mistranslated somehow.
This is what I've been using in docker-compose:
The is on Turbo 1.0.23 and 0.0.8.
I'm a little rusty on my Docker and haven't used Go before but I couldn't see anything obviously wrong in either.
Is it possible to show a working example of a docker-compose wrapped around tapico-turborepo-remote-cache that allows the host (eg MacOS) to run turbo against it successfully? Thanks!