search

facebook / pyre-check

Performant type-checking for python.
https://pyre-check.org/
MIT License
6.8k stars 433 forks source link

`pyre` crashes with `ƛ [Errno 86] Bad CPU type in executable: '/.venv/bin/pyre.bin'` on arm64 macOS #614

Open Bobronium opened 2 years ago

Bobronium commented 2 years ago

Pyre Bug

Bug description pyre crashes with ƛ [Errno 86] Bad CPU type in executable: 'venv/bin/pyre.bin'

Reproduction steps

$ mkdir my_project && cd my_project
$ python3 -m venv ~/.venvs/venv
$ source ~/.venvs/venv/bin/activate
(venv) $ pip install pyre-check
(venv) $ pyre init
(venv) $ pyre
Reproduction output ```shell ~/dev/temp $ mkdir my_project && cd my_project ~/dev/temp/my_project $ python3 -m venv ~/.venvs/venv ~/dev/temp/my_project $ source ~/.venvs/venv/bin/activate ~/dev/temp/my_project (venv) $ python -c "from platform import platform; print(platform())" macOS-12.4-arm64-arm-64bit ~/dev/temp/my_project (venv) $ pip install pyre-check Collecting pyre-check Using cached pyre_check-0.9.13-py3-none-any.whl Collecting psutil Using cached psutil-5.9.1-cp310-cp310-macosx_12_0_arm64.whl Collecting async-generator Using cached async_generator-1.10-py3-none-any.whl (18 kB) Collecting intervaltree Using cached intervaltree-3.1.0-py2.py3-none-any.whl Collecting pyre-extensions Using cached pyre_extensions-0.0.27-py3-none-any.whl (12 kB) Collecting testslide>=2.7.0 Using cached TestSlide-2.7.0-py3-none-any.whl Collecting typing-extensions Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB) Collecting tabulate Using cached tabulate-0.8.9-py3-none-any.whl (25 kB) Collecting libcst Using cached libcst-0.4.3-cp310-cp310-macosx_11_0_arm64.whl (1.8 MB) Collecting click>=8.0 Using cached click-8.1.3-py3-none-any.whl (96 kB) Collecting dataclasses-json Using cached dataclasses_json-0.5.7-py3-none-any.whl (25 kB) Collecting typeguard>=2.10.0 Using cached typeguard-2.13.3-py3-none-any.whl (17 kB) Collecting Pygments>=2.2.0 Using cached Pygments-2.12.0-py3-none-any.whl (1.1 MB) Collecting marshmallow<4.0.0,>=3.3.0 Using cached marshmallow-3.15.0-py3-none-any.whl (47 kB) Collecting marshmallow-enum<2.0.0,>=1.5.1 Using cached marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB) Collecting typing-inspect>=0.4.0 Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB) Collecting packaging Using cached packaging-21.3-py3-none-any.whl (40 kB) Collecting mypy-extensions>=0.3.0 Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB) Collecting sortedcontainers<3.0,>=2.0 Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting pyyaml>=5.2 Using cached PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl (173 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) Installing collected packages: pyparsing, packaging, typing-extensions, mypy-extensions, marshmallow, typing-inspect, typeguard, sortedcontainers, pyyaml, Pygments, psutil, marshmallow-enum, testslide, tabulate, pyre-extensions, libcst, intervaltree, dataclasses-json, click, async-generator, pyre-check Successfully installed Pygments-2.12.0 async-generator-1.10 click-8.1.3 dataclasses-json-0.5.7 intervaltree-3.1.0 libcst-0.4.3 marshmallow-3.15.0 marshmallow-enum-1.5.1 mypy-extensions-0.4.3 packaging-21.3 psutil-5.9.1 pyparsing-3.0.9 pyre-check-0.9.13 pyre-extensions-0.0.27 pyyaml-6.0 sortedcontainers-2.4.0 tabulate-0.8.9 testslide-2.7.0 typeguard-2.13.3 typing-extensions-4.2.0 typing-inspect-0.7.1 WARNING: You are using pip version 21.2.4; however, version 22.1.1 is available. You should consider upgrading via the '/Users/bobronium/.venvs/venv/bin/python3 -m pip install --upgrade pip' command. ~/dev/temp/my_project 6s (venv) $ pyre init ƛ Also initialize watchman in the current directory? [Y/n] ƛ Created basic `.watchmanconfig` at /Users/bobronium/dev/temp/my_project/.watchmanconfig ƛ Current directory is being watched by `watchman`. ƛ Which directory(ies) should pyre analyze? (Default: `.`): ƛ Successfully initialized pyre! You can view the configuration at `/Users/bobronium/dev/temp/my_project/.pyre_configuration`. You can now run the type checker with `pyre`. ~/dev/temp/my_project 17s (venv) $ pyre ƛ [Errno 86] Bad CPU type in executable: '/Users/bobronium/.venvs/venv/bin/pyre.bin' ```

Expected behavior Pure binary installed via pip is suitable for arm64

pyre rage output ```js Versions ======== Client version: 0.9.13 Could not determine binary version: [Errno 86] Bad CPU type in executable: '/Users/bobronium/.venvs/venv/bin/pyre.bin' Configuration ============= { "global_root": "/Users/bobronium/dev/temp/my_project", "dot_pyre_directory": "/Users/bobronium/dev/temp/my_project/.pyre", "disabled": false, "do_not_ignore_errors_in": [], "excludes": [], "extensions": [], "ignore_all_errors": [], "ignore_infer": [], "other_critical_files": [], "search_path": [], "site_package_search_strategy": "pep561", "site_roots": [], "source_directories": [ "SimpleRawElement(root='/Users/bobronium/dev/temp/my_project')" ], "strict": false, "taint_models_path": [ "/Users/bobronium/.venvs/venv/lib/pyre_check/taint" ], "use_buck2": false } Watchman Watch-list =================== { "version": "2022.05.16.00", "roots": [ "/Users/bobronium/dev/temp/my_project", "/Users/bobronium/dev/py/parametrize" ] } Server Log (2022-05-25 15:58:45.626231) ======================================= Client Log ========== 2022-05-25 15:58:45,617 [PID 72347] INFO Waiting for server... 2022-05-25 15:58:45,622 [PID 72347] INFO Cannot find a running Pyre server. Starting a new one... 2022-05-25 15:58:45,622 [PID 72347] INFO No binary specified, looking for `pyre.bin` in PATH 2022-05-25 15:58:45,622 [PID 72347] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 6. 2022-05-25 15:58:45,624 [PID 72347] INFO No typeshed specified, looking for it... 2022-05-25 15:58:45,624 [PID 72347] INFO Found: `/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed` 2022-05-25 15:58:45,625 [PID 72347] INFO Starting server at `my_project`... 2022-05-25 15:58:45,625 [PID 72347] INFO Writing arguments into /var/folders/sj/5p3z7pjd4qlb76z3d2vrjp3w0000gn/T/pyre_arguments_uohyumer.json... 2022-05-25 15:58:45,625 [PID 72347] DEBUG Arguments: { "source_paths": { "kind": "simple", "paths": [ "/Users/bobronium/dev/temp/my_project" ] }, "search_paths": [ "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$pyparsing", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$marshmallow", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$packaging", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$typeguard", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$click", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$libcst", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$testslide", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$dataclasses_json", "/Users/bobronium/.venvs/venv/lib/python3.10/site-packages$pip", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stdlib", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/DateTimeRange", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Deprecated", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Flask-Cors", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Flask-SQLAlchemy", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/JACK-Client", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Markdown", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Pillow", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/PyMySQL", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/PyYAML", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Pygments", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/SQLAlchemy", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/Send2Trash", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/aiofiles", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/annoy", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/appdirs", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/atomicwrites", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/aws-xray-sdk", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/babel", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/backports.ssl_match_hostname", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/beautifulsoup4", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/bleach", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/boto", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/braintree", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/cachetools", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/caldav", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/certifi", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/characteristic", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/chardet", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/chevron", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/click-spinner", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/colorama", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/commonmark", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/contextvars", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/croniter", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/cryptography", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/dataclasses", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/dateparser", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/decorator", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/docopt", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/docutils", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/editdistance", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/emoji", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/entrypoints", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/filelock", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/first", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-2020", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-bugbear", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-builtins", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-docstrings", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-plugin-utils", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-rst-docstrings", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-simplify", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/flake8-typing-imports", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/fpdf2", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/freezegun", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/frozendict", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/google-cloud-ndb", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/hdbcli", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/html5lib", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/httplib2", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/humanfriendly", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/invoke", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/jmespath", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/jsonschema", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/ldap3", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/mock", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/mypy-extensions", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/mysqlclient", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/oauthlib", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/openpyxl", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/opentracing", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/paramiko", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/parsimonious", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/passlib", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/passpy", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pep8-naming", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/playsound", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/polib", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/prettytable", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/protobuf", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/psutil", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/psycopg2", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyOpenSSL", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyRFC3339", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyaudio", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pycurl", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyfarmhash", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyflakes", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pynput", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pysftp", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pytest-lazy-fixture", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/python-dateutil", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/python-gflags", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/python-nmap", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/python-slugify", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pytz", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/pyvmomi", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/redis", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/regex", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/requests", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/retry", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/selenium", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/setuptools", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/simplejson", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/singledispatch", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/six", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/slumber", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/stdlib-list", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/stripe", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/tabulate", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/termcolor", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/toml", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/toposort", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/ttkthemes", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/typed-ast", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/tzlocal", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/ujson", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/urllib3", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/vobject", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/waitress", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/whatthepatch", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/xmltodict", "/Users/bobronium/.venvs/venv/lib/pyre_check/typeshed/stubs/xxhash" ], "excludes": [], "checked_directory_allowlist": [ "/Users/bobronium/dev/temp/my_project" ], "checked_directory_blocklist": [], "extensions": [], "log_path": "/Users/bobronium/dev/temp/my_project/.pyre", "global_root": "/Users/bobronium/dev/temp/my_project", "debug": false, "python_version": { "major": 3, "minor": 10, "micro": 1 }, "shared_memory": {}, "parallel": true, "number_of_workers": 6, "strict": false, "socket_path": "/var/folders/sj/5p3z7pjd4qlb76z3d2vrjp3w0000gn/T/pyre_server_16c462f25f41a53ed3d377dc23b286b7.sock", "show_error_traces": false, "additional_logging_sections": [ "server" ], "watchman_root": "/Users/bobronium/dev/temp/my_project", "taint_model_paths": [ "/Users/bobronium/.venvs/venv/lib/pyre_check/taint" ], "store_type_check_resolution": false, "critical_files": [ { "full_path": "/Users/bobronium/dev/temp/my_project/.pyre_configuration" } ] } 2022-05-25 15:58:45,628 [PID 72347] ERROR [Errno 86] Bad CPU type in executable: '/Users/bobronium/.venvs/venv/bin/pyre.bin' ```

Please run your reproduction steps followed by pyre rage > pyre_rage.log, and upload the file here:

pyre_rage.log

System information

grievejia commented 2 years ago

Do you have rosetta installed on your machine? If not, you can run softwareupdate --install-rosetta to install it.

Bobronium commented 2 years ago

No, I don't and I won't. It's an obvious workaround, but I'd prefer native arm64/apple silicon support.

grievejia commented 2 years ago

Including apple silicon binary in our release is indeed better, but I don't think the team has capacity to work on it in the short term.

I'll leave this issue open until someone in the team picks it up then. In the meantime, I think the only way around is to do a local build on M1.

Kludex commented 1 year ago

I'm building a codemod to migrate Pydantic from V1 to V2, and I intend to use TypeInferenceProvider from LibCST. Is there something I can do to speed up this? 👀

Kludex commented 1 year ago

I think the only way around is to do a local build on M1.

How do I build it? 😅

Rocamonde commented 1 year ago

Guys, do you really expect people to build their own binaries to use your type checker on over 50% of Apple laptops in current operation? Could you not add a CD hook to build the binaries? CircleCI seems to have support.

I have enjoyed using Pyre on my Linux environments over the last couple of weeks. Pyre looks like an attractive option for those developers looking to try something new. I anticipate that most newcomers to Pyre would come from Apple Silicon users, and so it seems like a valuable thing to support their development environment to continue growing the community.

At least, please add a notice to the readme for Apple Silicon users making clear what the installation procedure is and what the expectations are. This seems like a pretty major usability drawback that people should be aware of before deciding to use Pyre. I spent over half an hour on this because this is not well-documented anywhere.

Rocamonde commented 1 year ago

To be clear, I am more likely to install the OCaml compilation environment than Rosetta. And I hate adding bloat to my computer.