open-austin / indigent-defense-stats

A web scraper for collecting and processing public case records from sites using Tyler Technology's Odyssey court records database software.
MIT License
15 stars 5 forks source link

Update indigent-defense-stats code to reflect updates in azure-indigent-defense code #62

Closed nicolassaw closed 2 weeks ago

nicolassaw commented 2 weeks ago

Update the code from this repo's code to reflect the code of this repo.

Focus on these functions:

Worry less about these functions:

nicolassaw commented 2 weeks ago

I compared the versions of the different parts of the code (scraper, parser, additional tools, any additional cleanup functions, etc.) and found many folders and pieces of code were related to the Azure/LogicApps infrastructure.

Example: Scraper: First part of the scraper is initializing the blob container and receiving parameters from a single request sent via JSON (we aren't doing this with the VM). Most everything else looks the same besides the additions and changes to accommodate the LogicApps infrastructure that doesn't need to be copied over.

However, I also found an update in 2023 to the parser that added handling for "also known as". This was added to the current parser. Very helpful.

Otherwise, closing this ticket out unless @normaljosh has some other ideas of things I might have missed.

nicolassaw commented 2 weeks ago

I'm also copying over this piece of code (very important): "clean_case_json.py"