search

omnilib / aioitertools

itertools and builtins for AsyncIO and mixed iterables
https://aioitertools.omnilib.dev
MIT License
243 stars 24 forks source link

aioitertools.zip is doing strange things with exceptions #153

Open filip-zyzniewski opened 1 year ago

filip-zyzniewski commented 1 year ago

Description

What happens:

$ python3 -m venv aioitertools_zip_exception
$ cat aioitertools_zip_exception.py 
import asyncio
import platform
import sys

import aioitertools

async def gen():
    for i in range(10):
        if i == 9:
            assert False
        await asyncio.sleep(1)
        yield i

async def gen_pairs():
    for i in range(10):
        if i == 9:
            assert False
        await asyncio.sleep(1)
        yield i, "a"

async def main():
    print("platform version", platform.version())
    print("python version", sys.version)
    print("aioitertools version", aioitertools.__version__)

    print("the exception is getting swallowed:")
    async for (i, j) in aioitertools.zip(gen(), gen()):
        print(i, j)

    print("this attempts to unpack the exception:")
    async for (i, j), (k, l) in aioitertools.zip(gen_pairs(), gen_pairs()):
        print(i, j, k, l)

asyncio.run(main())
$ . ./aioitertools_zip_exception/bin/activate
(aioitertools_zip_exception) $ python3 -m pip install aioitertools
Collecting aioitertools
  Downloading aioitertools-0.11.0-py3-none-any.whl (23 kB)
Installing collected packages: aioitertools
Successfully installed aioitertools-0.11.0
(aioitertools_zip_exception) $ python3 aioitertools_zip_exception.py 
platform version Darwin Kernel Version 22.6.0: Wed Jul  5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000
python version 3.11.5 (main, Aug 24 2023, 15:09:45) [Clang 14.0.3 (clang-1403.0.22.14.1)]
aioitertools version 0.11.0
the exception is getting swallowed:
0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

this attempts to unpack the exception:
0 a 0 a
1 a 1 a
2 a 2 a
3 a 3 a
4 a 4 a
5 a 5 a
6 a 6 a
7 a 7 a
8 a 8 a
Traceback (most recent call last):
  File "/Users/Filip.Zyzniewski/aioitertools_zip_exception.py", line 38, in <module>
    asyncio.run(main())
  File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/Filip.Zyzniewski/aioitertools_zip_exception.py", line 34, in main
    async for (i, j), (k, l) in aioitertools.zip(gen_pairs(), gen_pairs()):
              ^^^^^^
TypeError: cannot unpack non-iterable AssertionError object
(aioitertools_zip_exception) $

What I expected to happen: I expected the exception to percolate up the call stack and be raised at the top level.

Details

filip-zyzniewski commented 1 year ago

It seems like this behavior got introduced in https://github.com/omnilib/aioitertools/commit/835bad734c297e7f4b36eda28d45be5c57290317 .

amyreese commented 9 months ago

@filip-zyzniewski can you confirm if #157 matches your expected behavior?

filip-zyzniewski commented 9 months ago

@amyreese not exactly, please see this:

$ pwd
/Users/Filip.Zyzniewski/git/github.com/amyreese/aioitertools
$ git status
On branch zip-exception
Your branch is up to date with 'origin/zip-exception'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    aioitertools_zip_exceptions.py

nothing added to commit but untracked files present (use "git add" to track)
$ git rev-parse HEAD                
6285944f42395ffdca96aa388d2db1e4f545c7bb
$ cat aioitertools_zip_exceptions.py    
import asyncio
import platform
import sys

import aioitertools

def gen():
    for i in range(2):
        yield ValueError(f"ValueError({i})")

async def agen():
    for i in range(2):
        yield ValueError(f"ValueError({i})")

async def main():
    print("platform version", platform.version())
    print("python version", sys.version)
    print("aioitertools:", aioitertools)
    print("aioitertools version", aioitertools.__version__)

    print("zip builtin behaves as expected:")
    for (i, j) in zip(range(10), gen()):
        print(i, j)

    print("aioitertools.zip raises an exception passed as a value:")
    async for (i, j) in aioitertools.zip(range(2), agen()):
        print(i, j)

asyncio.run(main())
$ python3 aioitertools_zip_exceptions.py
platform version Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000
python version 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)]
aioitertools: <module 'aioitertools' from '/Users/Filip.Zyzniewski/dev/git/github.com/amyreese/aioitertools/aioitertools/__init__.py'>
aioitertools version 0.11.0
zip builtin behaves as expected:
0 ValueError(0)
1 ValueError(1)
aioitertools.zip raises an exception passed as a value:
Traceback (most recent call last):
  File "/Users/Filip.Zyzniewski/dev/git/github.com/amyreese/aioitertools/aioitertools_zip_exceptions.py", line 31, in <module>
    asyncio.run(main())
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.7_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/Filip.Zyzniewski/dev/git/github.com/amyreese/aioitertools/aioitertools_zip_exceptions.py", line 27, in main
    async for (i, j) in aioitertools.zip(range(2), agen()):
  File "/Users/Filip.Zyzniewski/dev/git/github.com/amyreese/aioitertools/aioitertools/builtins.py", line 441, in zip
    raise v
ValueError: ValueError(0)
$