search

plazi / treatmentBank

Repository devoted to house keeping of treatmentBank
0 stars 0 forks source link

TB processing status indicator #35

Open myrmoteras opened 2 years ago

myrmoteras commented 2 years ago

@gsautter can we get an inidicator, webpage, whatever where we can check out the system health, capacity, planned work to be informed and plan accordingly. This would also be good to know in advance when such big jobs are being processed that affect daily work

see eg https://github.com/plazi/treatmentBank/issues/34

Tx

gsautter commented 2 years ago

Health is already monitored constantly, if not displayed on the website (the watchdog checks every 5 minutes).

There might be a way of showing the size of the various work queues, even though apart form super rare events like a full re-export of all our 50K+ DwC-As, they tend to be very low or empty ... page full of zeros for most of the time.

In general, I'd be comfortable with such a status indicator only if it's behind a login, as displaying such system internals publicly is somewhat of a security concern ... need to consult with Johannes regarding whether or not publicly displaying monitoring information really is a good idea.

Also, as such backlogging events are preciously rare, maybe we could go with an up-front announcement via mail for starters.

myrmoteras commented 2 years ago

HI Guido here is how GBIF is doing it: https://www.gbif.org/system-health

Can't we have something similar? If you do not want to have this public, then let's put a link here https://github.com/plazi/Plazi-Communications/wiki eg to the news or data access.

Regarding scheduling of jobs, lets add here an item maintenance or whateever you want to name it, and we will tell everybody to keep an eye on it.