sdb1228 / SoccerLCScrapers

Scrapers written in JS for Soccerlc
0 stars 0 forks source link

Batches v2: Batches_frd, the Batchening #9

Closed JoelHough closed 7 years ago

JoelHough commented 7 years ago

We need batches again. Add something like a lastTouchedByBatchAt timestamp to Game that has the Batch startedAt on it so we can do a Game.where(lastTouchedByBatchAt=batch.startedAt) to get all the games touched by that batch or a >= for games touch by that batch or later.

Maybe store the json intermediate db with a batch for debugging. Postgres has jsonb.

JoelHough commented 7 years ago

Games now have a lastBatchAt column that is touched with the scraper start time. That's good enough for now.