biglocalnews / court-scraper

Scrapers for U.S. county court sites.
ISC License
61 stars 18 forks source link

SeleniumHelpers broken for Odyssey #181

Open zstumgoren opened 1 year ago

zstumgoren commented 1 year ago

In the course of debugging #180, ran into the below error. It appears that in the course of abstracting SeleniumHelpers into a base class, we lost the ability to supply a web driver to Odyssey page objects (e.g. PortalPage, Search etc all require a driver on instances to function properly).

In order to avoid breaking high-level SeleniumHelpers, which appears to now be used in IA and WI scrapers, we should subclass SeleniumHelpers in odyssey namespace and update all references in Odyssey page classes to use the downstream version:

# court_scraper/platforms/odyssey/selenium_helpers.py

from court_scraper.base.selenium_helpers import SeleniumHelpers as BaseHelpers

class SeleniumHelpers(BaseHelpers):
    def __init__(self, driver):
        self.driver = driver

To reproduce the bug:

court-scraper search --with-browser -p wa_adams -c 08-2-00866-9

Executing search for wa_adams
Traceback (most recent call last):
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/bin/court-scraper", line 8, in <module>
    sys.exit(cli())
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/court_scraper/cli.py", line 74, in search
    results = runner.search(**kwargs)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/court_scraper/platforms/odyssey/runner.py", line 45, in search
    data = site.search(case_numbers=case_numbers)
  File "/Users/tumgoren/.local/share/virtualenvs/court-scraper-etl-B5T25nbt/lib/python3.7/site-packages/court_scraper/platforms/odyssey/site.py", line 55, in search
    portal_page = PortalPage(self.driver)
TypeError: PortalPage() takes no arguments
RandallJEllis commented 1 year ago

Has there been any progress on this?