search

sc3 / cookcountyjail

A Django app that tracks the population of Cook County Jail over time and summarizes trends.
http://cookcountyjail.recoveredfactory.net/api/1.0/?format=json
Other
31 stars 23 forks source link

Scraper fails as non-null field booking date is not properly handled #372

Closed nwinklareth closed 10 years ago

nwinklareth commented 10 years ago

As part of issue #355, in addition to changing the property of booking_date from datetime to date also changed it so it would not allow nulls, however this change causes this exception to be thrown:

The reason this happened was because I made an assumption about how null checking occurs and I did not do a test the code to.

This issue supercedes issue #367.

To fix it will require going back 15 days to catch missing booked persons.

bepetersn commented 10 years ago

Ugh. Did you do a migration? And if so, did South give you an option to supply a one-off value... Or maybe you just need to change the model field definition. I always get confused about what exactly needs to be done. On May 11, 2014 1:15 PM, "nwinklareth" notifications@github.com wrote:

As part of issue #355 https://github.com/sc3/cookcountyjail/issues/355, in addition to changing the property of booking_date from datetime to date also changed it so it would not allow nulls, however this change causes this exception to be thrown:

  • Exception is null value in column "booking_date" violates not-null constraint

The reason this happened was because I made an assumption about how null checking occurs and I did not do a test the code to.

This issue supercedes issue #367https://github.com/sc3/cookcountyjail/issues/367 .

To fix it will require going back 15 days to catch missing booked persons.

— Reply to this email directly or view it on GitHubhttps://github.com/sc3/cookcountyjail/issues/372 .

nwinklareth commented 10 years ago

Yes to both questions.

On Sun, May 11, 2014 at 1:20 PM, Brian Everett Peterson < notifications@github.com> wrote:

Ugh. Did you do a migration? And if so, did South give you an option to supply a one-off value... Or maybe you just need to change the model field definition. I always get confused about what exactly needs to be done. On May 11, 2014 1:15 PM, "nwinklareth" notifications@github.com wrote:

As part of issue #355 https://github.com/sc3/cookcountyjail/issues/355,

in addition to changing the property of booking_date from datetime to date also changed it so it would not allow nulls, however this change causes this exception to be thrown:

  • Exception is null value in column "booking_date" violates not-null constraint

The reason this happened was because I made an assumption about how null checking occurs and I did not do a test the code to.

This issue supercedes issue #367< https://github.com/sc3/cookcountyjail/issues/367> .

To fix it will require going back 15 days to catch missing booked persons.

— Reply to this email directly or view it on GitHub< https://github.com/sc3/cookcountyjail/issues/372> .

— Reply to this email directly or view it on GitHubhttps://github.com/sc3/cookcountyjail/issues/372#issuecomment-42778506 .

Regards

Norbert

Norbert Winklareth

bepetersn commented 10 years ago

Has the status of this changed after #372, or do we not know yet?

bepetersn commented 10 years ago

I pushed changes to the server for today that revert back to only searching 5 days in the past to look for missed inmates. Trusting Norbert's math, the scraper should be totally fine by now.