Signé Laval CMS not importing events from ville-de-laval

The logs on Datadog show that the aggregator stops after loading 43 events, but there are 144 events that should be loaded (with Artsdata IDs and without violations).

Why is the Aggregator stopping after loading 43 events?

Datadog https://app.datadoghq.com/logs?saved-view-id=1970191

Example event that should be loaded: Atelier d’écriture avec Danielle Godin http://kg.artsdata.ca/resource/K43-678

https://api.artsdata.ca/query?frame=event_footlight&format=json&sparql=https://raw.githubusercontent.com/culturecreates/footlight-aggregator/main/sparql/query-event-uri-v3.sparql&uri=https%3A%2F%2Fwww.laval.ca%2FPages%2FFr%2FCalendrier%2Fjournees-culture-biblio-arts-ecriture-danielle-godin.aspx%232024-09-27T23-00&source=http%3A%2F%2Fkg.artsdata.ca%2Fculture-creates%2Fartsdata-planet-ville-de-laval%2Fcalendrier-activites

Events that Maude is looking for in CMS:

We are encountering an issue with the Artsdata API. When fetching events from Artsdata, we use the source in batches. The aggregator calls the Artsdata API with an incremental offset count based on the selected batch size. The API calls stop if one batch has no results, assuming there are no more events.

The reason why it's not pulling all events from Ville de Laval is that the API call with offset=40&limit=5 is returning 3 events, and the call with offset=45&limit=5 is returning zero events. The iteration stops assuming no more events in the next batch. However, the call with offset=50&limit=5 is returning results.

Workaround: Changed the batch size to 10, and imported 144 events to CMS. Log: click here

Enhancement: If there is a way to determine the total count of minted events from the source, we can compare it with the imported events and log the results. This issue can be resolved by implementing the event-series iteration 3 proposal.

culturecreates / footlight-aggregator

Signé Laval CMS not importing events from ville-de-laval #128