alorenzo175
commented
4 years ago The job now exists and we can use the daily validation flag to see if something has been validated. What we still need is to check and validate older data (using /observations/values/unflagged) periodically.
Right now, daily validation is run via a cronjob on all observations. It might make more sense to schedule the long validation on an observation by observation basis to go from midnight to midnight, or extend the time window, etc. One way to do this would be to use https://github.com/rq/rq-scheduler. Since rq currently runs from the API, maybe a whole jobs framework in the API would make sense for this validation, fetching reference data, scheduled report generation, making reference forecasts.