Threaded refresh_matviews command, takes an optional --threads=N

[ironfroggy]

Answering questions in order:

• More faster is more better
• I did informal testing locally, but we'll want to do a few tests on a server to see how it affects RDS to find the sweet spot on thread count
• If we don't update in a transaction then the drilldown page could get partially updated sample resources or mismatches between the actual data and the counts, which are always updated last.
• I think celery tasks would make it harder to enforce the ordering. Some of the views have a hierarchy, because they are themselves querying other views, so care had to be taken to be both parallel and to ensure that order is followed.

[dpoirier]

I was thinking we wouldn't schedule the refresh tasks for the children until after the parent was done updating. Do the children need to have an order enforced amongst themselves?

[ironfroggy]

No, you're right there. I was just thinking of them all being on the queue together, but they don't need to be.

We don't have any other celery tasks here yet. Do you want to include it and rewrite this branch before merging it?

[dpoirier]

I'm torn on whether this is worth adding celery to the system. Maybe @vkurup and @philtuska can weigh in?

If we don't switch to celery, I'd like to have some tests for the partitioning function, just so we're confident we're not accidentally missing refreshing any views.

[vkurup]

I do think there's value in adding celery to our deployment (for this, and for the existing cronjobs), but I don't see an absolute need for it to be done prior to this PR, as long as we think it could easily be migrated to celery once it was set up.

[dpoirier]

Getting @philtuskha's input