search

CivicTechAtlanta / georgia-courtbot

Helping people remember to attend court to help break the cycle of fines and jail time
4 stars 5 forks source link

Bug: Scraper fails because of unavailable local issuer certificate #30

Closed abrie closed 2 years ago

abrie commented 2 years ago

The scraper returns the following stack trace:

Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/scraper/__main__.py", line 47, in <module>
    cli()
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/scraper/__main__.py", line 24, in scrape
    scraper.data.dekalb_scraper.run(output, days)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/scraper/data/dekalb_scraper.py", line 165, in run
    results = scrape(days=days)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/scraper/data/dekalb_scraper.py", line 124, in scrape
    for officer in scraper.get_all_judicial_officers():
  File "/Users/anvil/development/codeforatl/georgia-courtbot/scraper/data/dekalb_scraper.py", line 19, in get_all_judicial_officers
    response = self.session.get(url, headers=self.headers)  # verify=False)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/anvil/development/codeforatl/georgia-courtbot/.venv/lib/python3.9/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='ody.dekalbcountyga.gov', port=443): Max retries exceeded with url: /portal/Home/Dashboard/26 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))