ODNZSL / nzsl-online

New Zealand Sign Language Dictionary
GNU General Public License v3.0
40 stars 11 forks source link

Always update the dictionary when a process starts in a deployed environment #1507

Closed joshmcarthur closed 1 year ago

joshmcarthur commented 1 year ago

In development and test, we restart processes all the time - fetching the dictionary database is unnecessary overhead, especially when we can update it on-demand.

In staging and production, processes tend to start and then stay running. For this reason, it's best that we update the dictionary every time these processes start, rather than waiting for other rules to be met.

We tried to handle this assuming the dictionary wouldn't exist, but it ends up baked into the Heroku slug, since asset precompilation starts the app. This stops the update from running until it's older than 4 weeks, which isn't ideal if we're trying to get a database fix in.

G-Rath commented 1 year ago

We tried to handle this assuming the dictionary wouldn't exist, but it ends up baked into the Heroku slug, since asset precompilation starts the app

I don't remember this, but I think ideally it would be best to actually remove it because the slug is already over the soft-limit and the bigger it gets the higher the risk that booting can timeout (and iirc the db is at least ~60mb?) - we should be able to run a command after assets:precompile right?

joshmcarthur commented 1 year ago

@G-Rath have you got an idea how to do this? AFAIK we can't determine which commands are run during slug compilation without buildpacks, and I'd rather not hang this specifically off asset compilation since any app boot writes the database.

I looked into using .slugignore for this, but this is applied after pushing, but before building, so it won't actually have any effect - the database will still be there after building

joshmcarthur commented 1 year ago

I should say I also looked into release commands, but these are just dynos run after deployment, it doesn't affect what gets compiled into the slug.

joshmcarthur commented 1 year ago

The DB is around 7mb, so it's not huge.

G-Rath commented 1 year ago

The DB is around 7mb, so it's not huge.

ok I was clearly misremembering then 😅

have you got an idea how to do this? AFAIK we can't determine which commands are run during slug compilation without buildpacks, and I'd rather not hang this specifically off asset compilation since any app boot writes the database.

in that case (+ cause its only 7mb not like 70mb), this is fine