JailTracker Database Project

vvthompkins commented 6 years ago

What problem are we trying to solve?

An accurate understanding of jail populations and recidivism rates is key to understanding mass incarceration in the US, and developing evidence-based policies that promote criminal justice reforms. Counties typically lack research departments that can aggregate and report on jail data, however many counties do publish up to date custody records online through various tools such as JailTracker. This project aims to scrape and aggregate historical and current jail records available through JailTracker county websites, creating a database that can be used for future analysis.

Who will benefit (directly and indirectly) from this project?

Directly, criminal justice researchers and academics will benefit from having a direct and current source of data that can be used to better understand what is happening in jails across the country, from calculating recidivism rates to the percent of people incarcerated for inability to pay fees or fines. Activists and advocacy groups will benefit from the progress that is made as the result of analyzing this data, and will be better able to identify and support policies that address the true nature of the problem of mass incarceration. Criminal justice system-involved individuals will also benefit over time, as reform efforts continue.

Where can we find any research/data available/articles?

The JailTracker website hosts a list of counties that use their tool. Here is a direct link to the JailTracker for Adair County, Kentucky.

The most recent report from the Bureau of Justice Statistics jail survey, which provides some additional context.

ProPublica undertook a similar type of project in Cook County, Illinois last year.

What help is needed at this time?

I have experience with HTML web scrapers, however JailTracker uses JavaScript and I would be grateful for assistance or advice as I expand my skills set. Any and all help is welcome!

What are the next steps (validation, research, coding, design)?

Depending on who is able to help out, there's a bit more research to be done, but I think it's almost at the coding stage.

How can we contact you outside of Github (list social media or places you're present)?

Twitter: @vienna_valencia Slack: vienna

Project management

Checklist for NEW ideas :baby:

Hey, you're official! You're now part of the growing Progressive HackNight community. Here's a few things to get started (a couple you've probably already done).

[x] Create this idea issue
[x] Flesh out the who, where, and what questions above
[x] Start the conversation about this new idea on Slack (this project uses #jailtracker )
[x] Respond with at least one update on this issue within the next month

Checklist for ACTIVE projects :fire:

Let's get this project started! When this idea starts taking off, the Progressive HackNight Team will start helping this project's lead(s) out with project management and connecting you to resources you may need. To get there, please complete and check off the following:

[ ] Post an update at least once a month to this issue. Use BASEDEF for ideas, but it's ok even if your update is just "nothing new happened this month" or "we saw a small increase in traffic to our app this month". If there's no activity for two months, that's no problem, life happens. We'll just label this as backlog so others know you'll get back to it when you have the time. If nobody hears from you at all in more than two months, we may mark it as abandoned so that others can pick up this idea and run with it.
[ ] Create a GitHub repository and Slack channel for work.
[ ] Create issues to describe each task that you plan to do or need help with and how a contributor can get started on that task. You might start and stop a lot, so consider issues as your to-do list.
[ ] Create a team for your core contributors
- This will make it easier for you to manage your github repo access. People on a team have the same level of access. Admin access will allow your trusted contributors to make changes as needed.
- You can remove and add people to your team as needed.
- Note: You can also allow collaborators outside of your team and give them more limited access.
[x] Create a Google Drive, Dropbox, or other cloud storage to share larger files
- Github and Data.World are good for code and data, respectively, especially when you need version control. But they're not good for very large files, documentation, articles, etc. A cloud storage option will allow you to easily share, create, and collaborate on documents with your team and help organize ideas and thoughts.
- Doing this early on can help your team stay organized and to onboard new contributors who wouldn't have access to files you all have shared over email.

If you get stuck at any point, feel free to reach out to the leadership team with an email to steering@progressivehacknight.org or come find an organizer at a HackNight. We're here to help bring great ideas to life!

onlyunusedname commented 6 years ago

Hello. I'm not really a GitHub user (as you could see from my profile. :) I’m afraid I’m just a selflishly clone-to-desktop lurker ) I recently joined Progressive Hack Night group but I haven't actually attended anything yet. Not sure how to get word to you. If this is messy and/or a lapse in GitHub protocol, please feel quite free to delete it.

I don’t know what stack you are using for this project, but I’m pretty sure I could either: a) build a way to scrape JailTracker sites; or b) point you in the direction of doing it yourself. The optimal solution may vary based on your stack, but in a nutshell, you want to find a scriptable headless browser. Look perhaps at PhantomJS and CasperJS. Also, Google maintains Headless Chromium and there is a supercool GitHub repo here: https://github.com/graphcool/chromeless which can (amongst other things) run Headless Chrome on AWS Lamba, which is a pretty great platform-agnostic solution.

Anyway, good luck with this project. It sounds like a potentially very useful resource for good things!

vvthompkins commented 6 years ago

Thanks @onlyunusedname!

Will you be at the next HackNight on Tuesday? I plan on attending and getting to work on this - I'm also on the Slack (vienna) so feel free to shoot me a message if you're in there too.

@schuyler1d was kind enough to put together this gist to get me started and also raised a concern regarding potential session quotas, suggesting that it might be worth reaching out to JailTracker for permission given the purpose is research. I'm hoping to decide on a plan of action and get everything set up next week, and I'd be more than happy to have your help!

clayms commented 6 years ago

jailtracker base url https://jailtracker.com/jtclientweb/(S(mulnvzadhnvdszy32kyare55))/jailtracker/index/ e.g. for Midland County MI https://jailtracker.com/jtclientweb/(S(mulnvzadhnvdszy32kyare55))/jailtracker/index/Midland_County_MI

moajzashahab commented 6 years ago

Goggle Sheet: https://docs.google.com/spreadsheets/d/1R0TdnXLoUWr7dWNVWQGGC43Mk1C0CEk4enYdtkfKesw/edit?usp=sharing

ibrand commented 5 years ago

This project had some great progress made, and @viennav told us a little while ago that she couldn't support it at that time. This would be a great project to pick up if there is a Python lead looking to own it.

vvthompkins commented 5 years ago

This project had some great progress made, and @viennav told us a little while ago that she couldn't support it at that time. This would be a great project to pick up if there is a Python lead looking to own it.

Sorry for the lack of updates. I'm still working on this on the side even though I've moved out of state - the nonprofit where I work now is considering picking this project up, which I'm open to as long as the de-identified data is made publicly available.

ibrand commented 5 years ago

Sounds good. If a python lead comes along who's interested we can connect you all, but otherwise good luck and let us know if we can support you!

vvthompkins commented 5 years ago

Thanks @ibrand!

eenblam commented 4 years ago

Hi! Not sure if anyone in this thread is still working on related issues, but I just pushed out JTT, a simple JailTracker tracker. I had previously written it for bfeldman89/jail_scrapers, but we decided to break the project out so that folks outside of Mississippi could leverage the JailTracker component.

It's still a very simple bit of code, but it works! Right now it's only good for dumping JSON, but I'll be adding some additional utilities for different storage options (AirTable and SQLite) shortly. Contributions welcome! :)

ProgressiveHackNight / project-ideas