[BML-10]Update BillML repo to parse all bill data and create, upload full datasets - Githubissues

unitedstates / congress

Public domain data collectors for the work of Congress, including legislation, amendments, and votes.

https://github.com/unitedstates/congress/wiki

Creative Commons Zero v1.0 Universal

929 stars 202 forks source link

[BML-10]Update BillML repo to parse all bill data and create, upload full datasets #298

Closed BorodaUA closed 1 year ago

BorodaUA commented 1 year ago

Hi 👋 team, please review changes that I made in scope of following task(s):

[BML-10]Update BillML repo to parse all bill data and create, upload full datasets

Here what is done:

added cutom logger
refactored project structure so it can be included as sub repo in https://github.com/dreamproit/BillML project
speed up bill status and bill text downloads with multiprocessing
added CLI script to show info about local bills data and missing bill status, bill text data with ability to download missing data(very handy after project initial downloads)

TODO:

use black, isort, flake8 for whole project after this PR review.