.appimage is slow to start — include .pyc into the bundle

[ValdikSS]

AppImage (Linux) version of electrum is rather slow to start, it takes about 3 seconds on my modern Intel 1340P machine. This is because Python bytecode (.pyc files) are not included into .appimage, and since appimage could not be updated, Python compiles them every run.

If .appimage is exctracted with ./electrum-4.5.5-x86_64.AppImage --appimage-extract and ran from the directory (so it could update itself), subsequent runs are vastly faster.

$ time ./electrum-4.5.5-x86_64.AppImage daemon --help
…
real    0m1,597s
user    0m1,201s
sys     0m0,075s

$ time ./electrum-4.5.5-x86_64.AppImage daemon --help
…
real    0m0,640s
user    0m0,380s
sys     0m0,052s

Please run the application once in your building process, and include all the built .pyc files into .appimage.`

[ValdikSS]

The other appimage-related optimization is to use zstd compression. Current "stable" version of AppImageKit does not support it, so new appimagetool should be used https://github.com/AppImage/appimagetool/

[SomberNight]

Sounds like it might be worth trying including the .pyc files indeed. In fact in the android apk I think we are only including the .pyc files for the same (startup speed) reasons.

Hopefully there is a nicer approach than actually "running" the application. There should be, as we are not doing that for Android.

[ValdikSS]

Yes, seems there's compileall module. python -m compileall ., something like that.

[SomberNight]

Note: I've just spent a day fixing reproducibility of our apks due to apparently .pyc files not being reproducible. (ref https://github.com/spesmilo/python-for-android/commit/0ab0d872e6c6b88ddc05b9c4ba6fcd3aa7921242)

The cherrypicked patches are part of cpython 3.11 fortunately: https://github.com/python/cpython/pull/27926 https://github.com/python/cpython/pull/8226

However I read through lots of discussions and patches, and these comments from core developers are not exactly reassuring: https://github.com/python/cpython/issues/78274#issuecomment-2007069213

Since this is no longer needed by CPython, I recommend closing the issue (and PR).

Do you mean that the initial issue was fixed? PYC are now reproducible?

No. I mean that they don't need to be reproducible.

Ok, but this issue is about making PYC files reproducible to reproducible builds.

Wheel doesn't contain pyc files. So Python ecosystem around PyPI doesn't need reproducible pyc.

It makes me somewhat uncomfortable with putting pycs in binaries...