Remove pyinstaller dependancy

[mbuckaway]

Is your feature request related to a problem? Please describe. pyinstaller has limitations, and currently, only supports Python 3.7 with Python 3.9 currently in the works. In alot of ways, this is old technology.

Describe the solution you'd like Discovered that using with using a virtual environment, one can deploy an application without having to "compile" python and related code into a binary. So far, I've tested this situation on Linux and I have been able to package ALL of the dependancies into the AppImage including the C libs. For Linux, this means we can use the latest python version and it should run on the oldest Linux versions because it has no external depandancies. This should mean targeting ARM platforms should be a matter of selecting ARM as a build system and not have to rely on pyinstaller's support. On other platforms, it means that we are distributing python and the scripts in a package, again, without having to use pyinstaller.

One of the benefits of this change would be the current python binary can be shipped with the package, whatever that happens to be.

[mbuckaway]

A rework of the CallupSeedingMgr build shows that pyinstaller can indeed be replaced with a virtualenv. Managed to make an AppImage for Linux with ALL the dependancies built in including running Python 3.8. This is something that should also work on MacOSX. Windows might require some convincing.

The virtualenv happens to include all the python code as well as any C Lib dependancies required by the python code, so there is no need to figure out which libs to copy. Some testing is still required to see is this process can help solve the problem of supporting older Linux versions.

[esitarski]

Agree that a virtual env is a viable approach on Linux and MacOSX. It leads to much faster updates as only the app code is downloaded (previous dependencies are already loaded). The core problem with Windows is that Python is not normally installed.

PyOxidizer is new and makes big claims. Interesting?

[mbuckaway]

Um, virtualenv "installs" python and everything needed to run the app into the virtualenv. There is no need to have python installed ever. Pyinstall, PyOxidixer, somebody's binary tool, yada, all have limitations.

The current AppImage (ie. virtualenv) I created has no external dependancies.

[mbuckaway]

Did some more testing on Linux. It turns out this will be a bit more involved than expect. In some ways, I've creating my own version of pyinstaller, just not compiling the code.

The steps seem to be:

• setup the venv into dist/usr (dist becomes effectively a chroot jail)
• replace the python symlink with the real python exe
• zip up the /usr/lib/python3.8 directory (or whatever version), and place it in dist/usr/lib/python3.8.zip. The zip is optional, but reduces the amount of space used
• copy all the clib dependancies on Python into dist/lib. Some work my be required to get any external dependancies from the venv into this directory, but so far, it appears venv takes care of it
• update dist/AppRun with a new PYTHONHOME set to APPDIR/usr and a correct PYTHONPATH. PYTHONPATH has to include both the /usr/lib/python3.8 and /usr/lib/python3.8/site-dependancies directories or things like wx can't be imported.
• pack up the whole mess into an appimage, and ship it.

For Linux, at least, the idea is to create an AppImage that will run on any Linux. For Mac and Windows, I've leave out the clib files, and just demand the user run a recent OS version. For Windows the directory names would reflect the default used. For UNIX, everything is /usr. Some work would be required to find out what windows python sets up for the sys.path by default.

This will work, but will takes some time to complete.

[stuartlynne]

I am getting excellent results with Nuitka and github

It takes a bit longer to package, as it converts Python to C and compiles that.

The result is a solid exe that can run standalone in Windows.

The resulting exe is faster than interpreted Python, although generally not as fast as C/C++.

Nuitka has been around for since about 2012, and is under active development. Currently supporting up to Python 3.11.

One limitation is that while it can produce binaries for Windows, Linux and MacOS, it cannot cross-compile. You need to build Windows under Windows, Linux under Linux, and MacOs on a Mac.

[esitarski]

Historically there have been a lot of problems with Nuitka and wxPython. I also don't expect major speedups with CrossMgr and Nuitka as most of the heavy lifting is done in NumPy, which is C++ already. Of course, I am always interested in a better solution and will look at this again.

On Mon, Apr 8, 2024 at 3:34 PM Stuart Lynne @.***> wrote:

I am getting excellent results with Nuitka https://nuitka.net/ and github https://github.com/Nuitka/Nuitka

It takes a bit longer to package, as it converts Python to C and compiles that.

The result is a solid exe that can run standalone in Windows.

The resulting exe is faster than interpreted Python, although generally not as fast as C/C++.

Nuitka has been around for since about 2012, and is under active development. Currently supporting up to Python 3.11.

One limitation is that while it can produce binaries for Windows, Linux and MacOS, it cannot cross-compile. You need to build Windows under Windows, Linux under Linux, and MacOs on a Mac.

— Reply to this email directly, view it on GitHub https://github.com/esitarski/CrossMgr/issues/55#issuecomment-2043509763, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABGXKNDLPYF2BOG2NBUZLDY4LWN5AVCNFSM4K2KWA7KU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBUGM2TAOJXGYZQ . You are receiving this because you commented.Message ID: @.***>

--

Edward Sitarski