search

codeforboston / clean-slate-data

MIT License
27 stars 13 forks source link

Look into the availability of NJ data as a proxy for MA #104

Closed mikemahoney218 closed 4 years ago

mikemahoney218 commented 4 years ago

We are attempting to find states with publicly available data in support of our effort to determine how many people are barred from sealing or expunging their records due to multiple offenses. While our last attempt to identify a proxy landed on Pennsylvania, we've found mixed evidence about whether or not PA is the best fit as a proxy state for MA -- see this comment for more information on that. As such, we're examining several states which may be better fits in order to see if any provide the data we need and are better fits than PA.

One of the states identified as better fits (see issue #31, PR #52) is New Jersey. As such, we should see if we can get sufficient data from New Jersey to allow us to approach this problem.

For this state to be a better proxy than PA, it must:

  1. Have a digital repository of criminal offense records
  2. Contain records of individual offenses (ideally, one record per offense)
  3. Have a method to identify offenders and track them across multiple cases
  4. Be in a format that we can obtain via web scrapers, without payment or human intervention

The first step of this task will involve identifying the location of criminal offense records online; you'll then need to do some sleuthing to figure out if we're able to hit whatever API serves the records without going through captcha's or paying money. Please update this issue with any progress you make!

mikemahoney218 commented 4 years ago

There exists the ability to search dockets on https://portal.njcourts.gov/webe4/ExternalPGPA/, but hidden behind two layers of captcha's which logs you out automatically after six minutes. Could spend more time looking for the API endpoint though, in the event they don't protect that once you've auth'd in (but I'd be surprised); I'm going to call this closed complete unless anyone extremely wants to dive into it.