freelawproject / courtlistener

A fully-searchable and accessible archive of court data including growing repositories of opinions, oral arguments, judges, judicial financial records, and federal filings.
https://www.courtlistener.com
Other
551 stars 151 forks source link

Microservice ReadError during free opinion import #4680

Open sentry-io[bot] opened 1 week ago

sentry-io[bot] commented 1 week ago

Three of these incidents so far. Can we catch them with a retry and have them succeed?

Sentry Issue: COURTLISTENER-8KV

ReadError: 
(7 additional frame(s) were not displayed)
...
  File "httpcore/_async/http11.py", line 186, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "httpcore/_async/http11.py", line 224, in _receive_event
    data = await self._network_stream.read(
  File "httpcore/_backends/anyio.py", line 32, in read
    with map_exceptions(exc_map):
  File "contextlib.py", line 162, in __exit__
    self.gen.throw(value)
  File "httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc

ReadError: 
(9 additional frame(s) were not displayed)
...
  File "cl/corpus_importer/tasks.py", line 707, in get_and_process_free_pdf
    success, msg = update_rd_metadata(
  File "cl/corpus_importer/tasks.py", line 2229, in update_rd_metadata
    response = async_to_sync(microservice)(
  File "cl/lib/microservice_utils.py", line 102, in microservice
    return await client.send(req)

Filed by @mlissner

quevon24 commented 1 week ago

hmm, we have autoretry_for set for this exceptions: ConnectionError, ReadTimeout, RedisConnectionError in get_and_process_free_pdf task

We could add the ReadError to retry on that specific exception.

mlissner commented 1 week ago

Sounds right to me.