mlissner
commented
7 years ago This turned out to be quite easy. I did a quick test of the items above:
from collections import Counter
c = Counter()
for row in rows:
rd = RECAPDocument.objects.filter(
pacer_doc_id=row.pacer_doc_id,
docket_entry__docket__pacer_case_id=row.pacer_case_id,
docket_entry__docket__court_id=map_pacer_to_cl_id(row.court_id)
)
count = rd.count()
if count == 1:
c.update(['one'])
elif count > 1:
c.update(['more'])
elif count < 1:
c.update(['zero'])Which created the following stats:
Counter({'one': 103850, 'zero': 4680})Right now I'm processing the 103k with:
for row in rows:
rd = RECAPDocument.objects.filter(
pacer_doc_id=row.pacer_doc_id,
docket_entry__docket__pacer_case_id=row.pacer_case_id,
docket_entry__docket__court_id=map_pacer_to_cl_id(row.court_id)
)
count = rd.count()
if count == 1:
rd[0].is_free_on_pacer = True
rd[0].save()
row.delete()When that's done, I'll dig into the remaining 4680 items.
As discussed in https://github.com/freelawproject/courtlistener/issues/657#issuecomment-313547723, when our downloader of free pacer documents finished, it had 109k items that it couldn't download because they were already in our DB:
These need to be reviewed and if they're valid we can: