Open ro-savage opened 3 years ago
Okay, I have not faced this particular problem with hasura but that's how other services in docker containers behave as well, so I am familiar with the problem.
My usually solution is to make the application more resilient during setup. i.e
assume you have only 2 containers, your_app
container and graphql-engine
container.
graphql-engine
.your_app
your_app
's setup scripts checks for dependenciesgraphql-engine
and waits for a response.... it gets 400
.your_app
image waits for 5 seconds and sends the request again... hurray, 200
.It's a simple logic but in future, say your_app
starts using... rabbitmq or some custom dependency, then you'll need just send request to rabbitmq / dependency and check the same thing before you tell you app to really get started taking requests.
A little verbose response but I hope it helps you in the meanwhile. I'll subscribe to this thread as well to see what team hasura thinks about the problem! :smile:
@atb00ker - Thanks for your input.
Unfortunately this won't work on Heroku. Heroku automatically decides when an app is live based on binding and re-routes traffic automatically.
This is basically what happens right now, we keep retrying until we get a 200 OK. Which isn't too bad, but its annoying that every deploy our error and alert detection starts going nuts. Plus it causes unnessary downtime.
If it didn't bind until the migration and other booting processes were finished, then it would mean there was 0 downtime.
@ro-savage I completely understand you. As far as I know the binding part is done by docker, not sure what Hasura team can do about it, but if you find an alternative solution, please let me know as well; might be useful for future projects. 😄
Whenever we do a new deploy of hasura, our app 'crashes' on heroku.
We believe this is due to the fact that the hasura image is binding to the dyno's port before hasura is ready to serve. Therefore Heroku will start redirecting traffic to the hasura even though hasura is still applying metadata and booting.
We've spoken to Heroku about this, and they said dynos will also start serve as soon as the port is binded.
Is there a command to start hasura and apply metadata, before it binds to the port / starts serving traffic? Or could the docker image be updated to only bind after hasura is ready to serve?
Here are our logs when booting - It basically just continues with the boot process but we get
at=error code=H10 desc="App crashed"
repeatedly for about 2-3 minutes until it's ready.