openstates / openstates-scrapers

source for Open States scrapers
https://openstates.org
GNU General Public License v3.0
846 stars 464 forks source link

MA failing since at least 2018-06-15 #2376

Closed openstates-bot closed 6 years ago

openstates-bot commented 6 years ago

MA has been failing since 2018-06-15

Based on automated runs it appears that MA has not run successfully in 2 days (2018-06-15).

  04:49:37 INFO pupa: No vote found in supplement for vote #372
04:49:37 INFO pupa: save bill H4599 in 190th as bill_9471289a-714a-11e8-8ced-02a7180ff528.json
04:49:39 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4490
04:49:39 INFO pupa: save bill H4490 in 190th as bill_961ce6f2-714a-11e8-8ced-02a7180ff528.json
04:49:43 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4494
04:49:43 INFO pupa: save bill H4494 in 190th as bill_985bd4d2-714a-11e8-8ced-02a7180ff528.json
04:49:46 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4495
04:49:46 INFO pupa: save bill H4495 in 190th as bill_9a2ce9fe-714a-11e8-8ced-02a7180ff528.json
04:49:49 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4485
04:49:49 WARNING pupa: Couldn't find title for H4485; skipping
04:49:54 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4493
04:49:54 INFO pupa: save bill H4493 in 190th as bill_9ed5be9a-714a-11e8-8ced-02a7180ff528.json
04:49:58 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4491
04:49:58 INFO pupa: save bill H4491 in 190th as bill_a12e3d98-714a-11e8-8ced-02a7180ff528.json
04:50:01 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4492
04:50:01 INFO pupa: save bill H4492 in 190th as bill_a3045a08-714a-11e8-8ced-02a7180ff528.json
04:50:04 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4487
04:50:04 WARNING pupa: Couldn't find title for H4487; skipping
04:50:07 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4605
04:50:08 INFO pupa: save bill H4605 in 190th as bill_a6b5ca7e-714a-11e8-8ced-02a7180ff528.json
04:50:10 INFO pupa: GET (with `requests`) - https://malegislature.gov/Bills/190/H4606
loaded Open States pupa settings...
ma (scrape, import)
  people: {}
  committees: {}
  bills: {}
Traceback (most recent call last):
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 111, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 47, in scrape
    yield from self.scrape_chamber('lower', session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 66, in scrape_chamber
    yield from self.scrape_bill(session, bill, chamber)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 168, in scrape_bill
    self.scrape_actions(bill, bill_url, session)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 198, in scrape_actions
    self.scrape_action_page(bill, page)
  File "/opt/openstates/openstates/openstates/ma/bills.py", line 232, in scrape_action_page
    y = int(action_name_content[1].split('YEAS')[0])
IndexError: list index out of range

Visit http://bobsled.openstates.org for more info.

showerst commented 6 years ago

Fix for this in Testing, may take a few hours since MA is a slow scrape.