search

hacsoc / the_jolly_advisor

Get Jolly!
http://advise.case.edu/
MIT License
15 stars 21 forks source link

Add a healthcheck to postgres #265

Closed ajm188 closed 7 years ago

ajm188 commented 7 years ago

The relevant step is the test:prepare and spec:prepare tasks. We "enhance" them (rake-speak) to do the custom wait_for_db task. This depends on environment so that the database config is loaded.

Note that we only do this "enhancement" if we're in docker, which we detect through the use of an environment variable which is set in the compose file.

ajm188 commented 7 years ago

Well it appears I was able to hook into the right place, as this looped for 10 minutes before travis killed it. Maybe I'm hooking in too early, so we're trying to connect to the database before creating it? I'll continue to dig.

ajm188 commented 7 years ago

Hmm I'm not convinced though. I wish I could come up with a way to more reliably repro

ajm188 commented 7 years ago

Got a repro. The "database does not exist" is a docker cache issue. The rake db:test:prepare CMD inside the container is what creates the database, so on a second build, docker views this layer as cached, so the command doesn't actually get run against the new db container.

So, moving the cmd out of the dockerfile and explicitly running it in the make target is a step in the right direction. I think then that we can remove the ActiveRecord::NoDatabaseError from our rescue block in connect_to_db

ajm188 commented 7 years ago

I came up with a hack (stopping the database container, and changing DB_HOST to be localhost) to simulate the scenario where postgres isn't ready for connections yet. This branch appears to do what it says on the tin: https://pastebin.com/4mjUR1zF

Going to clean up the commits, then this will be ready for review.

ajm188 commented 7 years ago

@matthewbentley this is ready for review. Thoughts? I'm not sure we should loop forever, and rely on travis to kill us after 10 minutes, since locally this would require a user to ctrl-C the build.