itamarst / eliot

Eliot: the logging system that tells you *why* it happened
https://eliot.readthedocs.io
Apache License 2.0
1.09k stars 65 forks source link

Provide a way to make a generator cooperate with eliot context nicely. #472

Open tomprince opened 3 years ago

tomprince commented 3 years ago

There is the private eliot_friendly_generator_function, which is almost what I need, but it initializes the context on the first call of next/send, rather than on creation.

My use cases is wrapping a generator provided to twisted's Cooperator.

itamarst commented 3 years ago

Just for context, that code was originally written by @exarkun and I don't actually quite understand it, though I'm sure I could with some effort.

Some options:

  1. Make that API public and have the functionality you want.
  2. Make a variant of Cooperator that has a context-per-iterator using contextvars (https://docs.python.org/3/library/contextvars.html). Eliot automatically derives its own context from that, e.g. for asyncio, so it should Just Work.
  3. No doubt there are other options.

My preference is item 2, since making generic context system is what contextvars is for, and I imagine it might be more broadly useful as a feature. And it's probably only a few lines of code:

  1. Create a Context per iterator.
  2. When __next__ is called on iterator, wrap it with context.run().

Something like:

from contextvars import Context
from twisted.internet.task import Cooperator

class IteratorWrapper:
    def __init__(self, it):
        self.it = it
        self.context = Context()
    def __next__(self):
        return self.context.run(self.it.__next__)

class CooperatorWithContext:
    def __init__(self):
        self._coop = Cooperator()
    def coiterate(self, iterator):
        return self._coop.coiterate(IteratorWrapper(iterator))
    # ... wrap other methods too ...
itamarst commented 3 years ago

Other notes:

  1. This seems like it could also be upstreamed into Twisted.
  2. If you're stuck on Python 2, note that I'm not doing any further Python 2 development on Eliot, it's Python 3 only at this point.