openactive-archive / conformance-services

Harvests and normalises OpenActive Opportunity feeds to a common representation
MIT License
0 stars 0 forks source link

Generating scheduled events two weeks ahead #93

Closed rhiaro closed 4 years ago

rhiaro commented 4 years ago

According to common/good practice, we generate events from a schedule up to two weeks into the future.

Note about this from @odscjames:

If an thing is published for 2 months starting 1st Aug on the 1st Aug, we get the data 2nd Aug and make events till 15th Aug, and then it's never updated again ..... won't the system never generate normalised events for the rest of Aug and Sept?

To resolve this we could:

a) Not have a restriction, and just generate all events from a schedule (but with a sensible bigger cap, for schedules with no count or endDate). b) Not have a restriction, and just generate all events from a schedule into the database, but limit it to two weeks into the future output from our RPDE feed. (Problem: how do we know which events came from a schedule? Do we implement this restriction for everything?) c) Re-run the NormaliseSchedulePipe on its own, ahem, schedule, every two weeks.

odscjames commented 4 years ago

I was thinking about C too. Something where we generate events 30 days ahead (or something) but turn the "normalised" boolean on the raw data table into a date/time column and use it to to re-normalise things every 15 days (or something). Could maybe have the pipes return a flag so we only do that to relevant events?

rhiaro commented 4 years ago

@thill-odi says current practice is for publishers to regularly keep their schedule updated, so new events should get triggered by this. So nothing extra to do here, closing.