search

ooni / data

OONI Data CLI and Pipeline v5
https://docs.ooni.org/data
8 stars 4 forks source link

Implement workaround for OTel spans exceptions #70

Open hellais opened 5 months ago

hellais commented 5 months ago

Sporadically we see this happening:

Traceback (most recent call last):
  File "/home/art/repos/ooni/data/oonipipeline/.venv/lib/python3.11/site-packages/opentelemetry/context/__init__.py", line 163, in detach
    _RUNTIME_CONTEXT.detach(token)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/art/repos/ooni/data/oonipipeline/.venv/lib/python3.11/site-packages/opentelemetry/context/contextvars_context.py", line 50, in detach
    self._current_context.reset(token)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: <Token var=<ContextVar name='current_context' default={} at 0x7f18a5587bf0> at 0x7f171ce74b00> was created in a different Context

There is an open ticket about this on temporal: https://github.com/temporalio/sdk-python/issues/441 and a recommended workaround here: https://github.com/open-telemetry/opentelemetry-python/issues/2606#issuecomment-1789515786

We should look into implementing it