Pyre returning ERROR Check command exited with non-zero return code: 1 when running with poetry

[frederiksteiner]

Pyre Bug

Bug description Pyre returning ERROR Check command exited with non-zero return code: 1 since newest release when using pyre with poetry and running the following command: poetry run pyre --noninteractive --debug --sequential check Was working with version 0.9.19.

Reproduction steps https://github.com/frederiksteiner/pyre-bug/tree/master Here would be a pretty small github repo, with all the steps in the read me or also here: Step 1: Install python 3.11.9 and poetry Step 2: Run "poetry env use 3.11" Step 3: Run "poetry install" Step 4: Run "poetry run pyre --noninteractive --debug --sequential check"

Expected behavior Running smoothly and returning the expected

Logs Please include any relevant logs here:

2024-05-17 10:34:20,118 [PID 27089] INFO No binary specified, looking for `pyre.bin` in PATH
2024-05-17 10:34:20,119 [PID 27089] INFO Pyre binary is located at `/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/bin/pyre.bin`
2024-05-17 10:34:20,119 [PID 27089] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 7.
2024-05-17 10:34:20,120 [PID 27089] INFO No typeshed specified, looking for it...
2024-05-17 10:34:20,120 [PID 27089] INFO Found: `/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed`
2024-05-17 10:34:20,120 [PID 27089] INFO Writing arguments into /tmp/pyre_arguments_aw85m6yb.json...
2024-05-17 10:34:20,120 [PID 27089] DEBUG Arguments:
{
  "source_paths": {
    "kind": "simple",
    "paths": [
      "/home/fs/code/bug-pyre/src",
      "/home/fs/code/bug-pyre$tests"
    ]
  },
  "search_paths": [
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/python3.11/site-packages",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stdlib",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/ExifRead",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/Pillow",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/PyMySQL",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/PyYAML",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/aiofiles",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/boto",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/chevron",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/colorama",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/ldap3",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/mysqlclient",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/paramiko",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/psycopg2",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/pycurl",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/python-dateutil",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/pytz",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/regex",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/requests",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/retry",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/tqdm",
    "/home/fs/.cache/pypoetry/virtualenvs/bug-pyre-9tZeV5hq-py3.11/lib/pyre_check/typeshed/stubs/ujson"
  ],
  "excludes": [],
  "checked_directory_allowlist": [
    "/home/fs/code/bug-pyre/tests",
    "/home/fs/code/bug-pyre/src"
  ],
  "checked_directory_blocklist": [],
  "extensions": [],
  "log_path": "/home/fs/code/bug-pyre/.pyre",
  "global_root": "/home/fs/code/bug-pyre",
  "debug": true,
  "python_version": {
    "major": 3,
    "minor": 11,
    "micro": 9
  },
  "shared_memory": {},
  "parallel": false,
  "number_of_workers": 7,
  "additional_logging_sections": [
    "-progress"
  ],
  "show_error_traces": false,
  "strict": true
}
2024-05-17 10:34:20,122 [PID 27089] ERROR Check command exited with non-zero return code: 1.

Please run your reproduction steps followed by pyre rage > pyre_rage.log, and upload the file here: pyre_rage.log

Additional context Add any other context about the problem here. (like dependencies in your venv, third party stub files being used, overall goals, etc.)

[stroxler]

Hi @frederiksteiner thanks for the bug report!

Unfortunately there's not enough in the logs for me to have a very good guess the problem. A couple questions for you:

• What operating system are you on?
• Does running pyre --sequential check help any? If so, it's likely a failure in our parallelization logic.

One possibility is that we made some changes around scheduling workers (Pyre runs in parallel by default) that might be playing a role here; if we see that --sequential is okay then this is an especially likely root cause.

[frederiksteiner]

• What operating system are you on? I am running WSL2 with Ubuntu 20.04.5. Same error happens, when running pyre in a docker container (redhat/ubi8-minimal:8.9)
• Does running pyre --sequential check help any? If so, it's likely a failure in our parallelization logic. Nope, unfortunately, the error is the same

[vthemelis]

I'm having the same issue. I cannot replicate reliably in some environments but never get it on others.

0.9.19 works but 0.9.21 doesn't. --sequential doesn't work.

@stroxler, could you provide the tags for versions 0.9.19 onwards? I don't see them on GitHub. Also, can you reliably rebuild the wheel for these versions?

[stroxler]

The version tags actually do exist, but for some historical reason I have no context on we prepend a v so the tags are v0.9.19 and v0.9.21; I can work on getting rid of the prepending but you can see the tags here: https://github.com/facebook/pyre-check/tags

The wheels can be reliably built in theory, although we might have some trouble actually accessing them (they are built by some internal CI processes that want to push the results to pypi); there's room for improvement here but it's what we have for now.

[vthemelis]

@frederiksteiner I cannot repro with your repo and steps on Mac. Do you have a dockerfile that can reproduce the issue?

[frederiksteiner]

@vthemelis I added a Dockerfile to the github repo: https://github.com/frederiksteiner/pyre-bug/blob/master/Dockerfile

It's not the most beautiful dockerfile, but it should do the job I guess :)

[vthemelis]

Looks like the issue is that pyre requires glibc>=2.34 (?).

# /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin 
/root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin)
/root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin: /lib64/libc.so.6: version `GLIBC_2.29' not found (required by /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin)
/root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin)
/root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin)
/root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /root/.cache/pypoetry/virtualenvs/bug-pyre-il7asoJj-py3.11/bin/pyre.bin)

[vthemelis]

@frederiksteiner, I guess that using a newer version of your operating system or upgrading glibc will resolve this for you. Also, I don't think that this is related to poetry.

@stroxler, would it be possible to statically link libc in pyre?

[frederiksteiner]

@vthemelis Yes that seems to be the case. When changing the image version to ubi9-9.4 everything seems to work properly

Should I close the issue?

[vthemelis]

I think that it would still be worth thinking about statically linking libc into the executable as part of the issue.

[stroxler]

Let's leave it open for now, I probably can't get an answer on statically linking this week but I'll ask around; that would probably be a great idea assuming there isn't something preventing it.

Also @vthemelis how did you get those logs? @samwgoldman had a theory that maybe OCaml is buffering stderr and then dying before it flushes, so I was wondering if we should look into it but I'm curious what worked for you.

[vthemelis]

@stroxler I got the logs by running pyre.bin directly (no arguments needed). The logs were in stderr and I opened #863 to propagate them through the python wrapper script.

[vthemelis]

I would look into the static linking bit but I don't have a Linux workstation handy which makes this a bit more tedious.

[vthemelis]

I had a quick look and in the end, I think that it may be a bad idea to statically link libc as this may make pyre even less compatible.

Looks like the best way is to use a manylinux container to build the wheel. I wonder if pyre could be build with such a container that supports an older version of libc.

[stroxler]

Yeah, I think we came to the same conclusion.

Our current setup uses internal CI infra (which does not use or allow Docker) to build Pyre so we'll need to translate all of that into a setup we can containerize.

[frederiksteiner]

Should I wait until #863 is done or should I close the issue now?

[stroxler]

We can close this, I'll keep working with @vthemelis to get #863 merged but I don't think that blocks this.

And I'll actually file a new issue for the glibc build since it's relatively clear what we need to do (and it may take some time)