vxgmichel / aiostream

Generator-based operators for asynchronous iteration
http://aiostream.readthedocs.io
GNU General Public License v3.0
801 stars 34 forks source link

Feature suggestion: `strict` parameter for `zip` #118

Closed smheidrich closed 4 months ago

smheidrich commented 4 months ago

Python 3.10 introduced the strict parameter for the built-in zip function, which, when set to True, can help spot bugs by raising an exception when the supplied iterables turn out to have different lengths (i.e. when one raises StopIteration after fewer iterations than the others).

I think it would be useful if aiostream's zip function supported this parameter as well.

Extra context: The strict parameter of Python's built-in zip turned out to be useful enough that linters like Ruff or flake8 with the bugbear plugin even have a rule to complain when you neither use it nor explicitly disable it by setting strict=False.

vxgmichel commented 4 months ago

Hi @smheidrich, thanks for the suggestion :+1:

I think it would be useful too, feel free to create a PR if you feel like contributing to the project. The implementation is over there: https://github.com/vxgmichel/aiostream/blob/5e0547afcebfcb556e011df3c185f69794a187cb/aiostream/stream/combine.py#L49-L86

It relies on asyncio.gather which exposes a return_exceptions argument. However, I don't think we can use that since we still want to propagate the exception as soon as possible if it is not a StopAsyncIteration. Instead, we probably want to turn StopAsyncIteration into a sentinel with a dedicated function:

async_stop_iteration_sentinel = object()
if strict:
    async def next_item(streamer):
        try:
            return await anext(streamer)
        except StopAsyncIteration:
            return async_stop_iteration_sentinel 
else:
    next_item = anext

[...]
try:
    coros = builtins.map(next_item, streamers)
    items = await asyncio.gather(*coros)
except StopAsyncIteration:
    break
if strict:
    if all(item == async_stop_iteration_sentinel for item in items):
        break
    if any(item == async_stop_iteration_sentinel for item in items):
        raise ValueError
[...]    

Otherwise, I'll do it myself when I find the time :)

vxgmichel commented 4 months ago

Done and released in v0.6.2 :tada:

Thanks a lot @smheidrich for your valuable contribution !

smheidrich commented 4 months ago

Great!! Thanks for the reviews & merge! :+1: