search

openstates / issues

Having trouble? Looking to contribute? Issues live here!
15 stars 2 forks source link

NV-scrape failing since at least 2021-01-14 #231

Closed openstates-bot closed 3 years ago

openstates-bot commented 3 years ago

NV-scrape has failed 5 times since 2021-01-14

Logs:

12:01:32 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=SCR&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:33 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=SR&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:34 INFO scrapelib: GET - http://www.leg.state.nv.us/Session/81st2021/Reports/TablesAndIndex/2021_81-index.html
12:01:35 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=AB&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:36 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=IP&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:37 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=AJR&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:38 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=ACR&Filters.DisplayTitles=false&Filters.PageSize=2147483647
12:01:39 INFO scrapelib: GET - https://www.leg.state.nv.us/App/NELIS/REL/81st2021/HomeBill/BillsTab?Filters.SearchText=&Filters.SelectedBillTypes=AR&Filters.DisplayTitles=false&Filters.PageSize=2147483647
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/bin/os-update", line 8, in <module>
    sys.exit(main())
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 318, in main
    report = do_update(args, other, juris)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 205, in do_update
    report["scrape"] = do_scrape(juris, args, scrapers)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 89, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/scrape/base.py", line 173, in do_scrape
    "no objects returned from {} scrape".format(self.__class__.__name__)
openstates.exceptions.ScrapeError: no objects returned from NVBillScraper scrape
LauraBula commented 3 years ago

Has this been resolved? I am wondering if I need to have my team manually check the Bills we are following. Thank you, Laura

showerst commented 3 years ago

No, Nevada doesn't start until Feb 1 so we won't get this working until they start posting data so we can see the new website structure.

LauraBula commented 3 years ago

Makes sense! Thank you!

LauraBula commented 3 years ago

Checking in on this again. The session has now started, but the scrape does not seem to be picking up the events.

For example, on NV SB5 (https://www.leg.state.nv.us/App/NELIS/REL/81st2021/Bill/7180/Overview), the state has an event of 2/1/2021

image

However, the OpenStates feed for this bill does not have this event.
{ "id": "ocd-bill/f352c24e-803d-41bc-a4e8-186055e13a30", "session": "81", "jurisdiction": { "id": "ocd-jurisdiction/country:us/state:nv/government", "name": "Nevada", "classification": "state" }, "from_organization": { "id": "ocd-organization/c4400c7b-157d-442a-b688-543ffb989967", "name": "Senate", "classification": "upper" }, "identifier": "SB 5", "title": "Makes changes relating to telehealth. (BDR 40-416)", "classification": [ "bill" ], "subject": [ "INSURANCE", "COMPUTERS AND INFORMATION SYSTEMS (See also DIGITAL CONTENT AND DIGITAL TRANSACTIONS)", "MEDICAID", "BEHAVIORAL HEALTH", "LEGISLATURE", "FRATERNAL BENEFIT SOCIETIES", "HOSPITAL, MEDICAL AND DENTAL SERVICE CORPORATIONS, NONPROFIT", "HEALTH AND HUMAN SERVICES, DEPARTMENT OF", "AUDIO-VISUAL COMMUNICATIONS", "HEALTH CARE PROVIDERS (See also specific providers)", "HEALTH MAINTENANCE ORGANIZATIONS", "MANAGED CARE", "INDUSTRIAL INSURANCE", "INTERNET (See also DIGITAL CONTENT AND DIGITAL TRANSACTIONS)", "DENTAL CARE, ORGANIZATIONS FOR", "TELEPHONE AND TELECOMMUNICATIONS", "BEHAVIORAL HEALTH, COMMISSION ON", "TELEHEALTH", "PATIENT PROTECTION COMMISSION" ], "extras": {}, "created_at": "2020-11-21T10:09:24.370063Z", "updated_at": "2021-01-13T18:12:57.647082Z", "openstates_url": "https://openstates.org/nv/bills/81/SB5/", "first_action_date": "2020-11-18T08:00:00Z", "latest_action_date": "2020-12-14T08:00:00Z", "latest_action_description": "From printer.", "latest_passage_date": null, "abstracts": null, "other_titles": null, "other_identifiers": null, "sponsorships": null, "actions": [ {

jamesturk commented 3 years ago

thanks for the nudge, it looks like we need at least a partial rewrite of NV based on their site changes, I'm on that now

jamesturk commented 3 years ago

this will be up tomorrow, one or two small things to spot check 👍

LauraBula commented 3 years ago

THANK YOU!!!!!!!!

Empowerment | Excellence | Teamwork Laura Carpenter, CEO [cid:image005.png@01D6FA48.AE59D340]

Office: 480.339.8383 Cell: 602.740.4058 2150 E Highland Ave., Suite 105 Phoenix, Arizona 85016​ www.bulalaw.comhttp://www.bulalaw.com [cid:image006.jpg@01D6FA48.AE59D340] This electronic mail transmission and any attachments are confidential. They should be read or retained only by the intended recipient. If you have received this transmission in error, please notify the sender immediately and delete the transmission from your system. BulaLaw provides general information about statutes and regulations. It does not provide legal advice

From: James Turk notifications@github.com Sent: Wednesday, February 3, 2021 4:03 PM To: openstates/issues issues@noreply.github.com Cc: Laura Carpenter laura.carpenter@bulalaw.com; Manual manual@noreply.github.com Subject: Re: [openstates/issues] NV-scrape failing since at least 2021-01-14 (#231)

this will be up tomorrow, one or two small things to spot check 👍

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/openstates/issues/issues/231#issuecomment-772886566, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASRF35DYACLMPMNUPK6OUHDS5HI3BANCNFSM4WCKNOTA.

openstates-bot commented 3 years ago

closed via successful run on 2021-02-04