The data for the ingestor is gathered via the JSON Stream's on F1's static site.
I've fixed the historical ingestor by including separate logic for updating streams. For example, the driver's list can have multiple updates pushed to it. This can be seen in the JSON Stream for the Sazuka Race where line 1 is basic driver data, then later followed by headshots on line 3.
This is a draft as there is one issue that isn't fixed with this PR. The real time ingestor still has this bug. I've written code to fix this, however, I can't test it at the moment. The code can be viewed on the dev-driver-data-fix-patch branch.
We can merge this PR without fixing the real time ingestor, but it will only fix already processed history. Additionally, it does not delete previous database entries so I had to scrap my local database and rerun the ingestor.
Fixes #42, fixes #43.
The data for the ingestor is gathered via the JSON Stream's on F1's static site.
I've fixed the historical ingestor by including separate logic for updating streams. For example, the driver's list can have multiple updates pushed to it. This can be seen in the JSON Stream for the Sazuka Race where line 1 is basic driver data, then later followed by headshots on line 3.
Compared to the sessions that have complete data, for example the Melborne Qualifying, all of the driver data is on one line.
This is a draft as there is one issue that isn't fixed with this PR. The real time ingestor still has this bug. I've written code to fix this, however, I can't test it at the moment. The code can be viewed on the dev-driver-data-fix-patch branch.
We can merge this PR without fixing the real time ingestor, but it will only fix already processed history. Additionally, it does not delete previous database entries so I had to scrap my local database and rerun the ingestor.