alexlydiate
commented
3 years ago Hi Sylvan,
As per a recent email from Zimmerman, we understand that Zimmerman don't wish to continue with the integration between iati.cloud and our IATI Validator - "We will disconnect from the IATI (official) validator this week as we will make internal arrangements for data validation. We advise IATI to pick up data files from the IATI Registry in another way."
To avoid disruption to the Validator we've disabled that sync - which, incidentally, was handy, as we noted the Datastore APIs went down for about a day last week, which would have upset the state of the Validator - and we are working on an alternate manner to get our data. There will be a short period where new files are not available in the Validator, but that period isn't likely to be long enough to cause significant issue to publishers.
As of the end of play on 30th June we'll no longer be pointing our Datastore subdomain at iati.cloud and not too long after that the Validator will be getting its data from elsewhere - so, as now agreed, iati.cloud (and, clearly, the Datastore) and the Validator will no longer have a relationship.
That's the technical situation as it stands - for anything beyond that, we'll leave it outside of GitHub. Ta, and best of luck.
sylvanr
Hi guys!
Just a quick refresher for everyone reading: The process between the datastore and the validator was as follows: The datastore retrieves updated datasets from the IATI Registry. The validator periodically looks at the datastore, retrieving and validating datasets that have been updated. The datastore waits for this validation to finish, then proceeds to parse the updated datasets.
Some problems in the recent weeks and days: The datastore has been increasing it’s parsing speed, making the whole process take around a day on average, however, the validator API (accessed from), has been returning 502’s occasionally, leading to the process being delayed.
More recently, the validator has not been validating datasets, at least not visibly from the aforementioned endpoint. The most recent example is: the datastore downloaded new datasets on the 25th of June, the datastore waited until the 28th of June for the Validator to start and finish validating. The validator has not started in this time. This means that the datastore’s parsing proces has been fully disrupted.
As always, correct me if I am wrong! But in general we would like to continue parsing valid datasets!
Cheers, Sylvan