search

oils-for-unix / oils

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!
http://www.oilshell.org/
Other
2.84k stars 156 forks source link

'test/unit.sh all' may not work on Python 3.8 / Ubuntu 20.04 - AttributeError: module 'typing' has no attribute 'Iterable' #1014

Open malkarouri opened 2 years ago

malkarouri commented 2 years ago

Trying to follow Contributing on Ubuntu 20.04, with Python 2.7.18 and Python 3.8.10 installed. The command build/dev.sh ubuntu-deps fails as python-pip is not in Ubuntu 20.04 anymore so pip is installed manually. I am using the current code cloned today (commit b723e9af).

The issue is when running test/unit.sh minimal:

> test/unit.sh minimal // omitted output of successful tests // ... mycpp/format_strings_test.py] Traceback (most recent call last): File "mycpp/format_strings_test.py", line 8, in import unittest File "/usr/lib/python3.8/unittest/init.py", line 60, in from .async_case import IsolatedAsyncioTestCase File "/usr/lib/python3.8/unittest/async_case.py", line 1, in import asyncio File "/usr/lib/python3.8/asyncio/init.py", line 8, in from .base_events import * File "/usr/lib/python3.8/asyncio/base_events.py", line 45, in from . import staggered File "/usr/lib/python3.8/asyncio/staggered.py", line 16, in coro_fns: typing.Iterable[typing.Callable[[], typing.Awaitable]], AttributeError: module 'typing' has no attribute 'Iterable'

mycpp/format_strings_test.py FAILED

xargs: test/unit.sh: exited with status 255; aborting

real 0m4.366s user 0m3.597s sys 0m0.977s

It seems that mycpp/format_strings_test.py fails as it runs Python 3 but imports typing from vendor/typing instead of the standard library, and the latter does not have Iterable defined.

malkarouri commented 2 years ago

If Iterable is added manually to vendor/typing.py then the same test fails due to an AttribtuteError on Awaitable. If Awaitable is added then the test fails with a TypeError, because Iterable and Callable are defined as None and are not subscriptable.

andychu commented 2 years ago

Ah OK thanks for the report, I have seen this kind of thing before... the typing usage is a little subtle

Not sure what we should do about it at the moment.

I think maybe the easiest thing is to simply leave out the Python 3 tests like format_strings_test.py.

I guess it doesn't fail on my machine because I have Python 3.6 which has a different standard library! Doh!

For now I would just remove 'mycpp' from this line and I think everything should work? Let me know if it doesn't

https://github.com/oilshell/oil/blob/master/test/unit.sh#L47

I will think about if we should do that permanently

andychu commented 2 years ago

Oh I guess the issue is that PYTHONPATH should include vendor/ for the Python 2 stuff, but NOT Python 3.

That might be the better fix, but let me know if commenting it out works

malkarouri commented 2 years ago

Removing mycpp from test/unit.sh makes the step work. In fact, the smoke and osh-all test report success as well.

andychu commented 2 years ago

OK great! I just made a commit to automate this, so please sync to HEAD.

Let me know if you have any more problems and questions about Oil!

I usually start with bin/osh -c 'echo hi' and bin/osh -n -c 'echo hi' and hack from there :)

malkarouri commented 2 years ago

Just did. The test passes cleanly now. Closing..

Thanks a lot. That was impressive! I will continue exploring and hacking.

andychu commented 2 years ago

Glad to hear it!

I'll leave this open until test/unit.sh all works reliably, but at least test/unit.sh minimal works now