linkedin / shiv

shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.
BSD 2-Clause "Simplified" License
1.74k stars 94 forks source link

permission denied #86

Open ibeex opened 5 years ago

ibeex commented 5 years ago

doing shiv -e cpe_m.cpe_mi:cli -o my_command.pyz . in setup.py dir and then try to run command I get this strange error msg?

[I] ➜ ./my_command.pyz
zsh: permission denied: ./my_command.pyz
lorencarvalho commented 5 years ago

hi @ibeex

Unfortunately this is not enough information to troubleshoot, what are the permissions on that file? also, can you share the version of shiv you are using and show me head -n1 my_command.pyz?

thanks

ibeex commented 5 years ago

here it is

[I] ➜ ll my_command.pyz
-rwxr-xr-x 1 myuser users 642K Nov 10 16:21 my_command.pyz

[I] ➜ head -n1 my_command.pyz
#!/usr/share/bash-completion/completions/python3

[I] ➜ type python
python is /home/myuser/.pyenv/shims/python

[I] ➜ type shiv
shiv is /home/myuser/.local/bin/shiv

shiv is installed with pipsi, I user pyenv to manage python versions, and virtualenvs

Edit: it works using -p shiv -e cpe_middleware.cpe_middleware:cli -p '/usr/bin/env python3' -o my_command.pyz . is it bug or intended behaviour?

lorencarvalho commented 5 years ago

if I had to venture a guess I'd say the issue is the shebang there - is that a valid interpreter? does the pyz execute if you invoke it with your pyenv interpreter path? e.g.

/home/myuser/.pyenv/shims/python ./my_command.pyz

I'm not super familiar with pipsi, but nothing jumps out at me as incompatible with how shiv works.

ibeex commented 5 years ago

/home/myuser/.pyenv/shims/python ./my_command.pyz works, why is default python detected #!/usr/share/bash-completion/completions/python3 ?

lorencarvalho commented 5 years ago

That is an excellent question - I have no idea why that would be the case.

If, when building a pyz with shiv, you do not specify an interpreter or shebang (using the -p option), then shiv defaults to sys.executable but attempts first to suss out whether it's being invoked from a virtualenv or not, and if so, what is the actual location of that interpreter (rather than the interpreter that's been copied into the virtualenv).

This is done via the _interpreter_path function (found here)

How did you make your pyz? Were you in an activated virtualenv? Did you specify a shebang using -p? In order to help I need to know more about your environment, and if possible, steps to be able to reproduce. Thanks :)

ibeex commented 5 years ago

Hi @lorencarvalho my python setup involves pyenv pyenv manages python versions and virtualenviroments. My local pyton tools like black and shiv are installed in separate virtualenviroments with pipsi this configuration ensure that nothing is installed to system python and every project can haw separate python version and virtualenv.

So shiv is in separate virtualenv and it was invoked in virtualenviroment for my project shiv -e cpe_m.cpe_mi:cli -o my_command.pyz ..

One more thing as I understand shiv should make pyz on your local machine and simplify deployment on remote machine. So isn't better that default shebang should be /usr/bin/env python3 or /usr/bin/env python2 depending on python version. And _interpreter_path can return one or another depending is it py2 or py3.

lorencarvalho commented 5 years ago

@ibeex shiv only supports Python 3 (specifically 3.6, as f-strings are used) currently. It is theoretically possible that we could build Python 2 zipapps, but given EOL for Py2 is inching closer and closer every day (2020!) we've opted not to.

I'm unfamiliar with the implementation of pipsi and pyenv, so I'm not in the best position to debug those issues, but yes, if you intend to ship your pyz to another machine, then passing --python "/usr/bin/env python3" when creating your pyz is generally the best practice.

yarikoptic commented 5 years ago

I am having a similar issue with incorrect shebang:

gory details (click to expand) ```shell /tmp > virtualenv --system-site-packages --python=python3 venvs/dev3 Already using interpreter /usr/bin/python3 Using base prefix '/usr' New python executable in /tmp/venvs/dev3/bin/python3 Also creating executable in /tmp/venvs/dev3/bin/python Installing setuptools, pkg_resources, pip, wheel...done. /tmp > source venvs/dev3/bin/activate (dev3) /tmp > pip install shiv Collecting shiv Using cached https://files.pythonhosted.org/packages/66/cc/67b69bffa38e620cea0d211d74f19b30304c21e584dcd407da007b30c007/shiv-0.0.48-py3-none-any.whl Requirement already satisfied: pip>=9.0.3 in ./venvs/dev3/lib/python3.7/site-packages (from shiv) (19.2.3) Requirement already satisfied: setuptools in ./venvs/dev3/lib/python3.7/site-packages (from shiv) (41.2.0) Collecting click!=7.0,>=6.7 (from shiv) Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl Installing collected packages: click, shiv Found existing installation: Click 7.0 Not uninstalling click at /usr/lib/python3/dist-packages, outside environment /tmp/venvs/dev3 Can't uninstall 'Click'. No files were found to uninstall. Successfully installed click-6.7 shiv-0.0.48 (dev3) /tmp > shiv requests -o requests.pyz --quiet (dev3) /tmp > ./requests.pyz zsh: permission denied: ./requests.pyz (dev3) /tmp > head -n1 ./requests.pyz #!/usr/lib/python3 (dev3) /tmp > ls -l /usr/lib/python3 -d drwxr-xr-x 3 root root 4096 Oct 19 2014 /usr/lib/python3/ (dev3) /tmp > shiv --version shiv, version 0.0.48 ```
Specifying `-p '/usr/bin/env python3'` mitigates it ```shell /tmp > shiv requests -o requests.pyz -p '/usr/bin/env python3' --quiet (dev3) /tmp > ./requests.pyz Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> ```
jhermann commented 4 years ago
$ python3 -m this | grep guess
In the face of ambiguity, refuse the temptation to guess.

If you ask me, -p should be a mandatory option. ;)

Possibly allowing things like {pymajor} based on the running interpreter.