The first version of the Next NG version of the Scraper queries the V1 database to find out which booking_ids to scrape from the Sheriff's website. This dependency needs to be brokern. Currently the scraper needs to know the following set of bookings ids:
the set of active booking ids from the last scrape - this is the last known population
the set of potential booking ids for the days since the last scrape - this is the potential new booked inmates
the set of discharged booking ids for the last N days - this helps to find inmates that were marked as discharged, as no repsonse was received when they were scraped often due to network or system problems.
the set of not found booking ids for the last N days - this helps to find inmates that for one or reason or not were not found in earlier scrape, oftem due to network or system problems, howver may also be due to input issues at the Sheriff's site..
This is part of #395.
The first version of the Next NG version of the Scraper queries the V1 database to find out which booking_ids to scrape from the Sheriff's website. This dependency needs to be brokern. Currently the scraper needs to know the following set of bookings ids:
All of these booking ids can be determined from the raw Inmate data. For a discussion on how to do so is found on the Wiki page: [Notes On Working With Raw Inmate Data}(https://github.com/sc3/cookcountyjail/wiki/Notes-On-Working-With-Raw-Inmate-Data).