search

unitedstates / congress

Public domain data collectors for the work of Congress, including legislation, amendments, and votes.
https://github.com/unitedstates/congress/wiki
Creative Commons Zero v1.0 Universal
929 stars 202 forks source link

Error from lxml when parsing amendments "purpose" field #293

Closed achokey-crp closed 1 year ago

achokey-crp commented 1 year ago

Hi,

I've run into some errors when parsing the bulk data after running the following commands:

usc-run govinfo --bulkdata=BILLSTATUS --congress=117 
usc-run bills

Seems like there might be an issue with getting fields that don't exist. Here is the error, for example, when parsing hr51-117

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/site-packages/congress/tasks/utils.py", line 174, in process_set
    results = fetch_func(id, options, *extra_args)

  File "/usr/local/lib/python3.8/site-packages/congress/tasks/bills.py", line 148, in process_bill
    process_amendments(bill_id, xml_as_dict, options)

  File "/usr/local/lib/python3.8/site-packages/congress/tasks/bills.py", line 292, in process_amendments
    amendment_info.process_amendment(amdt, bill_id, options)

  File "/usr/local/lib/python3.8/site-packages/congress/tasks/amendment_info.py", line 13, in process_amendment
    amdt = build_amendment_json_dict(amdt_data, options)

  File "/usr/local/lib/python3.8/site-packages/congress/tasks/amendment_info.py", line 57, in build_amendment_json_dict
    'purpose': amdt_dict['purpose'][0] if type(amdt_dict['purpose']) is list else amdt_dict['purpose'],

KeyError: 'purpose'
JoshData commented 1 year ago

I think I've fixed it in 939bf01. Fingers crossed.