IA failing since at least 2018-01-04

[openstates-bot]

IA has been failing since 2018-01-04

Based on automated runs it appears that IA has not run successfully in 2 days (2018-01-04).

  00:01:18 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=123
00:01:19 INFO pupa: save person Joe Bolkcom as person_d904fb90-f1dd-11e7-a2d5-0242ac110005.json
00:01:19 INFO pupa: save membership d904fb90-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "upper"} as membership_d904ff1e-f1dd-11e7-a2d5-0242ac110005.json
00:01:19 INFO pupa: save membership d904fb90-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "party", "name": "Democratic"} as membership_d9050158-f1dd-11e7-a2d5-0242ac110005.json
00:01:19 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=18073
00:01:20 INFO pupa: save person Nate Boulton as person_d9a0ccb4-f1dd-11e7-a2d5-0242ac110005.json
00:01:20 INFO pupa: save membership d9a0ccb4-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "upper"} as membership_d9a0d07e-f1dd-11e7-a2d5-0242ac110005.json
00:01:20 INFO pupa: save membership d9a0ccb4-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "party", "name": "Democratic"} as membership_d9a0d2c2-f1dd-11e7-a2d5-0242ac110005.json
00:01:20 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=9665
00:01:21 INFO pupa: save person Tod R. Bowman as person_da30de30-f1dd-11e7-a2d5-0242ac110005.json
00:01:21 INFO pupa: save membership da30de30-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "upper"} as membership_da30e24a-f1dd-11e7-a2d5-0242ac110005.json
00:01:21 INFO pupa: save membership da30de30-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "party", "name": "Democratic"} as membership_da30e4a2-f1dd-11e7-a2d5-0242ac110005.json
00:01:21 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=10730
00:01:22 INFO pupa: save person Michael Breitbach as person_dacee788-f1dd-11e7-a2d5-0242ac110005.json
00:01:22 INFO pupa: save membership dacee788-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "upper"} as membership_daceec56-f1dd-11e7-a2d5-0242ac110005.json
00:01:22 INFO pupa: save membership dacee788-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "party", "name": "Republican"} as membership_daceeeb8-f1dd-11e7-a2d5-0242ac110005.json
00:01:22 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=18074
00:01:23 INFO pupa: save person Waylon Brown as person_db6751d0-f1dd-11e7-a2d5-0242ac110005.json
00:01:23 INFO pupa: save membership db6751d0-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "upper"} as membership_db675568-f1dd-11e7-a2d5-0242ac110005.json
00:01:23 INFO pupa: save membership db6751d0-f1dd-11e7-a2d5-0242ac110005 membership in ~{"classification": "party", "name": "Republican"} as membership_db6757a2-f1dd-11e7-a2d5-0242ac110005.json
00:01:23 INFO scrapelib: GET - https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=18041
no pupa_settings on path, using defaults
ia (scrape, import)
  people: {}
  bills: {}
  votes: {}
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 173, in do_scrape
  File "/opt/openstates/venv-pupa//bin/pupa", line 11, in <module>
    load_entry_point('pupa', 'console_scripts', 'pupa')()
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/__main__.py", line 67, in main
    subcommands[args.subcommand].handle(args, other)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 260, in handle
    return self.do_handle(args, other, juris)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/cli/commands/update.py", line 305, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
Traceback (most recent call last):
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/opt/openstates/venv-pupa/src/pupa/pupa/scrape/base.py", line 111, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/openstates/openstates/openstates/ia/people.py", line 13, in scrape
    yield from self.scrape_chamber('upper')
  File "/opt/openstates/openstates/openstates/ia/people.py", line 27, in scrape_chamber
    yield from self.scrape_member(chamber, link)
  File "/opt/openstates/openstates/openstates/ia/people.py", line 57, in scrape_member
    leg_page = lxml.html.fromstring(self.get(link.attrib['href']).text)
  File "/opt/openstates/venv-pupa/lib/python3.5/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/opt/openstates/venv-pupa/lib/python3.5/site-packages/scrapelib/__init__.py", line 292, in request
    raise HTTPError(resp)
scrapelib.HTTPError: 500 while retrieving https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=18041

Visit http://bobsled.openstates.org for more info.

[estaub]

The link to Senator Jim Carlin (https://www.legis.iowa.gov/legislators/legislator?ga=87&personID=18041 ), e.g. from https://www.legis.iowa.gov/legislators/find?address=&city=Le+Mars , is broken. Most/all other legislators seem fine.

I've notified the IA webmaster.

[estaub]

The "Jim Carlin" problem described above has been fixed; hopefully the scrape will succeed tonight.

[mileswwatkins]

Waiting on votes still; I'm in favor of turning off the vote scraper for now, will check with the rest of the team.