Bad interpreter on RHEL7.9 install

[esemplare]

Following instructions on https://www.pantsbuild.org/2.18/docs/getting-started/installing-pants, I'm unable to run pants after install. When running the script I get a bad interpreter: No such file or directory showing part of the shebang line cut off at 80 chars.

Looking into the pants script, the shebang line is very long, which it seems on this OS (RHEL7.9) exceeds the max length. Some google searches show it's a kernel specific configuration.

The shebang line reads /{USER_HOME}/.cache/nce/{PANTS_SHA}/venvs/2.19.0/bin/python3.9 -sE which can easily exceed 130 chars depending on the user name length.

Are there any workarounds?

[kaos]

There is a fix for this in pantsbuild/pex#2295 (pex v2.1.154)

This slack thread has info about how to use that fixed version of pex in pants where the default version is older: https://chat.pantsbuild.org/t/16320245/two-of-my-coworkers-have-been-bitten-by-the-long-shebang-iss

[esemplare]

Great, thank you. Appreciate your quick response!

[esemplare]

Prematurely closed this as I thought the solution as commented would work. It seems there is another individual who is posting in the chat a similar experience.

The latest scie-pants version has yet to be updated to use pex >= v2.1.154, so the latest scie-pants that gets installed from setup-pants.sh still has the issue.

[jsirois]

The version of Pex in question is decided by the Pants release process. The scie-pants install command uses that directly: https://github.com/pantsbuild/scie-pants/blob/2cd4278eb8bf773b398b88cdf6ef64fe5a4843bf/tools/src/scie_pants/install_pants.py#L75-L144

So, until the Pants release process builds the Pants PEX using a Pex version with the shebang ~fix, this problem will not be fixed. Another approach is to stop being a bystander and fix the scie-pants installer to not use a shell script (This has the advantage of fixing this issue for all releases of Pants retroactively). It could be saying /full/path/to/venv/bin/python /full/path/to/venv/bin/pants ... instead of using /full/path/to/venv/bin/pants .... Then there is no shebang in play at all since argv0 is a full-fledged binary (/full/path/to/venv/bin/python). This option has been available for a while but folks persistently seem to miss it. Alternatively, the scie-pants install step could itself emit a tiny #!/bin/sh script:

#!/bin/sh

exec /full/path/to/venv/bin/python /full/path/to/venv/bin/pants "$@"

This would allow the 2 current launch forks (native client vs non) to have the same single argv0 supplied via env var binding.

I will not be working on any of this, but bystanding has been painful; so I thought I'd inject some observations that appear to have escaped folks so far.

[esemplare]

Another approach is to stop being a bystander and fix the scie-pants installer to not use a shell script (This has the advantage of fixing this issue for all releases of Pants retroactively).

Not sure if I'm interpreting this incorrectly... by "bystander" you mean someone who is not taking part or just observing? Pretty toxic "observation" to say to someone who just researched a build tool and wants to evaluate it. To say the barrier of entry to using this software is contributing to the project is a first for me.

[jsirois]

@esemplare by bystander I mean the scie-pants project / maintainers offering the bad advice, not you. There were 2 levels of bad here:

1. Maintainers offering canned advice to upgrade Pex. That works for Pants runtime uses of Pex, but it doesn't work here where the problem is in scie-pants use of Pants PEXes. Basically people weren't actually thinking carefully about the problem you and others presented. Instead they were handing out canned advice that pattern matched loosely and you had to do the extra work to find out that advice was bad. In my opinion you should not have had to do that.
2. The idea scie-pants should just wait for a fixed Pex to solve its problems. It could have always instead understood the problem and fix that Pex had made for venv Python scripts and used that understanding to stop bystanding and instead take control here and fix proactively.

For me, both 1 & 2 were painful to watch happen and not happen, respectively; so I spoke up to hopefully stop the bleeding. I don't maintain the project any longer; so I won't be helping you directly, but hopefully @kaos or @thejcannon or some other active scie-pants maintainer can use the prompts to do so.

[thejcannon]

I appreciate the shift from "hostile towards observer" to "hostile towards helpful maintainer"

To have any hostility against members of our community who are volunteering their time and energy either by filing issues, trying to help users with those issues, or submitting fixes goes against our code of conduct.

---

@esemplare I appreciate you opening this issue and checking out Pants 👋

[esemplare]

@jsirois, appreciate the clarification and insight into the issues here 😄 . Definitely unsure of the history of this project, but I'm glad I misunderstood your sentiments.

[kaos]

@esemplare I too thank you for this ticket, and you evaluating the pants project.

What follows could be viewed as an internal matter to the project, to be had else where. I choose to post it here to have it in context, and for full transparency with our community.

---

@esemplare by bystander I mean the scie-pants project / maintainers offering the bad advice, not you. There were 2 levels of bad here:

1. Maintainers offering canned advice to upgrade Pex. That works for Pants runtime uses of Pex, but it doesn't work here where the problem is in scie-pants use of Pants PEXes. Basically people weren't actually thinking carefully about the problem you and others presented. Instead they were handing out canned advice that pattern matched loosely and you had to do the extra work to find out that advice was bad. In my opinion you should not have had to do that.
2. The idea scie-pants should just wait for a fixed Pex to solve its problems. It could have always instead understood the problem and fix that Pex had made for venv Python scripts and used that understanding to stop bystanding and instead take control here and fix proactively.

For me, both 1 & 2 were painful to watch happen and not happen, respectively; so I spoke up to hopefully stop the bleeding. I don't maintain the project any longer; so I won't be helping you directly, but hopefully @kaos or @thejcannon or some other active scie-pants maintainer can use the prompts to do so.

I highly value your insights, as always, @jsirois. In this case--clearly making these observations spurred by pained emotions were not delivered in a very helpful manner however. For me, most my "canned answers" come of this nature, I rarely have the time nor information to reproduce every question to verify my ideas/suggestions before posting them. Often times they do work however, in the cases they don't we all learn something from it and we (I/the pants community and the individual asking for help) can dig further to find a proper answer/solution.

I was not aware of the issue being in scie-pants, which completely invalidates point 2. As you point out you've stepped away as maintainer for pants/scie-pants; if it is painful for you to watch how we that remain carry the flag forward, I suggest you look away (or at least hold your reply to a time when the pain has faded)--as it seems the reaction we get from your pained observations are not very helpful (despite, I am sure, being intended as such, which is why I'm trying to show how your message was received with this reply.)

I've always valued the open welcoming and friendliness of the Pants community, where we all do our best with what we have. Remarks like "could have always instead understood the problem and fix that Pex had made for venv Python scripts and used that understanding to stop bystanding" to me are snarky at best, and something I fear goes in the opposite direction of those values.

fwiw, I did look into scie-pants use of pex following a reply made by the OP on slack, and posted a reply here about us upgrading pex in scie-pants--but consequently deleted the message as I hadn't have the time to fully understand how it fit into the picture, in fear of a message like the above, from you John. Having to second guess my partial conclusions for errors before sharing and feeling inadequate due to not investing enough time does not fill oneself with energy and motivation.

[jsirois]

I suggest you look away

Yeah, I agree. That has been hard to do due to my nature and the mechanical constraint of Pex being in the Pantsbuild org / using Slack. I've started this discussion to remedy that, afaict, last remaining part of the problem for me.

[esemplare]

Regarding this issue. It seems the latest release candidate (v2.20.0rc0) works. This can be closed.