search

hauntsaninja / mypy_primer

Run mypy and pyright over millions of lines of code
MIT License
55 stars 29 forks source link

Primer crashes with an unfriendly traceback if requirements for a project can't be installed #67

Closed AlexWaygood closed 1 year ago

AlexWaygood commented 1 year ago

When run on Python 3.11, primer currently crashes with the following traceback (see https://github.com/python/typeshed/pull/9336):

``` Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 996, in main retcode = inner() ^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 989, in inner retcode = asyncio.run(coro) ^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 768, in primer result = await result_fut ^^^^^^^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/asyncio/tasks.py", line 605, in _wait_for_one return f.result() # May raise f.exception(). ^^^^^^^^^^ File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 425, in primer_result await self.setup() File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 364, in setup await run( File "/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/site-packages/mypy_primer.py", line 112, in run raise subprocess.CalledProcessError(proc.returncode, cmd, output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '/tmp/mypy_primer/projects/_materialize_venv/bin/pip install -r ci/builder/requirements.txt' returned non-zero exit status 1. ```

This traceback arises from the fact that the requirements for the Materialize project can't be installed on Python 3.11. However, there's nothing in the traceback to indicate that it's the Materialize project that's the problem here (though admittedly it's not too hard to figure it out if you do a CTRL-F for ci/builder/requirements.txt in the mypy_primer source code).

Would you be open to a PR improving the error message if the requirements for a project can't be installed? Alternatively we could skip projects where the requirements can't be installed, but that seems like it could lead to errors going unnoticed, which would be bad.

JelleZijlstra commented 1 year ago

As a co-maintainer I'm going to say we definitely should make the error message better :)

We shouldn't silently skip projects that can't be installed, because then we may never notice when projects stop working. However, it might be reasonable to add some mechanism that allows us to skip Materialize if running under 3.11, if as typeshed maintainers we think the benefit of getting 3.11 coverage outweighs the cost of dropping this project.

hauntsaninja commented 1 year ago

Note if you were to run with 3.11, you may be affected by https://github.com/python/cpython/issues/100133