linkedin / shiv

shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.
BSD 2-Clause "Simplified" License
1.73k stars 94 forks source link

Ship the Python interpreter #32

Open alexprengere opened 6 years ago

alexprengere commented 6 years ago

Hello, For the generated zipapp to be fully standalone, is it technically possible to ship the full Python interpreter along? This way you could deploy your app on environments with different or no Python interpreter.

lorencarvalho commented 6 years ago

Hi @alexprengere,

Per the PEP, zipapps are zip files with a shebang at the top and a __main__.py file inside of them. This is the "magic" that allows the Python interpreter to be able to execute them. If the interpreter itself were included in the zipapp, it would no longer be a zipapp and would require some other mechanism to execute.

I believe there are some tools that ship the interpreter as well, pyinstaller comes to mind 🙂

devxpy commented 6 years ago

@sixninetynine You're absolutely correct! But pyinstaller is notoriously hard to configure compared to this.

This makes half of my deployment cycle (devops) a joke.

If it had this feature, I can't even comprehend the ease of deployment for django stuff..

Also the fact that this would enable us to ship python to mainstream users in a no-fuss way.

albertogomcas commented 4 years ago

Would there be a relatively easy way to combine shiv zips with portable distros like winpython zero?

lorencarvalho commented 4 years ago

hi @albertogomcas

I'm not familiar with winpython, but from a cursory look at their project page and documentation it appears they provide a statically compiled Python interpreter.

In theory, running a pyz created by shiv using the winpython interpreter should work just fine. All a user would need to do is specify the shebang (using shiv's -p flag) or invoke the pyz using the winpython interpreter (e.g. C:\\path\to\winpython.exe your_pyz).

warsaw commented 4 years ago

PyOxidizer is the tool I'm currently evaluating for this.

tovmeod commented 4 years ago

I tried to use PyOxidizer to run a shiv app unsuccessfully, did you manage to make them both work together?

k4ml commented 4 years ago

This article has some interesting story on shipping the interpreter. In short, it possible although with some caveats.

https://www.scylladb.com/2019/02/14/the-complex-path-for-a-simple-portable-python-interpreter-or-snakes-on-a-data-plane/

jhermann commented 4 years ago

At least on Linux, it is as easy as this:

# in a "shiv" working dir
$ tar xfz ~/tmp/egenix-pyrun-2.2.3-py3.5_ucs4-linux-x86_64.tgz ./bin/pyrun3.5
$ shiv -e shiv.cli:main -o shiv .
$ cat bin/pyrun3.5 shiv >bin/shiv
$ chmod +x ./bin/shiv 
$ ./bin/shiv 
…
  File "/home/jhe/src/github/shiv/bin/shiv/_bootstrap/__init__.py", line 86
    return root / f"{name}_{build_id}"
                                     ^
SyntaxError: invalid syntax

What happens here is that basically the bang path is replaced by an ELF binary which takes over execution of the appended ZIP. AFAIK, this also would work on Windows (given a compatible single-file release of Python).

Unfortunately PyRun currently only supports 3.5, and #124 prevents this from really working. Anyone know of other single-file Python releases?

ClericPy commented 4 years ago

Python really need a portable (or standalone / one-file) executable file for Interpreter, but I could hardly find a nice choice.

As I known, the only way to use python without the installing process is copy the whole python install-folder between same linux distributions, or the Windows embed portable files.

warsaw commented 4 years ago

I think PyOxidizer is the leading contender here.