datamade / how-to

📚 Doing all sorts of things, the DataMade way
MIT License
86 stars 12 forks source link

Scheduling tasks on Heroku #76

Closed jeancochrane closed 4 years ago

jeancochrane commented 4 years ago

Background

One of the last pieces that is keeping us from feeling confident in deploying complex apps on Heroku is that we don't yet know how to schedule jobs.

Proposal

I want to take a look at the Heroku Scheduler and related apps to define a way of scheduling tasks on Heroku.

Deliverables

I plan to update the Heroku documentation to include my guidance on how to do this.

Timeline

I expect this to take 2-3 R&D days.

jeancochrane commented 4 years ago

A first step I took here was to check if there was any simple alternative to Solr that we might want to explore first. I decided to trial Elastic App Search, a service by the creators of ElasticSearch that provides a developer-friendly API on top of an ElasticSearch instance along with a sleek dashboard for tuning the setup and an open source React template providing a quick search interface.

In general my impression was similar to #8: this is a really good idea that's not yet mature enough to put into production, but as we move toward incorporating React into more projects, we may want to keep an eye on it.

Specifically, some things I liked about Elastic App Search where:

Some things I didn't like:

Overall the idea is really solid and I'm brainstorming how we might steal the key ideas (templates for quickly setting up a basic search interface, fast deployments, well-tuned base configs) for our own Solr deployment work.

hancush commented 4 years ago

FYI, here are Heroku deployment artifacts for Metro: https://github.com/datamade/la-metro-councilmatic/pull/531/

jeancochrane commented 4 years ago

I'm going to reorient to make this issue about scheduling instead.

hancush commented 4 years ago

Here's the work I did/notes on scheduling: https://github.com/datamade/la-metro-councilmatic/issues/522