poll for app updates

[ghukill]

Scaffolding for polling to update Livy/Celery status in place, but thinking there are other updates this could request and trigger.

Hitting the route system_bg_status, where more information could be added. Pondering:

• status of Jobs
• global messages, could be removed when finished? or updated?

[ghukill]

For better or worse, good to keep in mind that Jobs (and likely other areas) are not updated until a page is refreshed. At the time, this was to avoid constantly pinging MySQL and/or Livy, and to avoid race conditions (likely more important).

The page refresh was the mechanism by which the status of a Job is updated.

[ghukill]

In exploring, became clear this would touch a lot of different parts. While polling for Livy/Celery status was done, and arguably beneficial, the approach would not support polling of statuses in other places.

Closing issue in favor of re-opening at some point to look at real-time updates and polling in a more holistic way.