Closed nihilox closed 2 years ago
Hello @nihilox,
Thank you for reporting this issue.
Unfortunately I see no good solution right now in order to recover 100% native Python/venv behaviour with AppImages. However, there is a simple workaround. The problem is actually related to -E
(i.e. ignoring PYTHON*
environment variables). If isolation is done manually, e.g. as:
PYTHONPATH= python -s -m site
then sys.path
is properly set.
The difficulty is that venv seems to rely on symbolic links, at least on Linux. My understanding is that venv site-packages location is determined from the link origin (i.e. sys.executable
). But, when using an AppImage, Python is not called directly, it is mounted at AppImage start to a temporary directory, and then it is called through an AppRun wrapper script. In order to mimic a direct call through a symbolic link, I use the exec command in the AppRun script, e.g. as
(exec -a "$origin" "$target" "$@")
where $target
is the actual Python runtime, inside the mounted AppImage. This properly sets sys.executable
to $origin
. However, when sys.executable
differs from $target
, then Python reads sys.base_prefix
from its (hardcoded) initial installation location, e.g. /usr
, which is no more valid once relocated, e.g. when using an AppImage. Therefore, I instead use the following command:
(PYTHONHOME="$base_prefix" exec -a "$origin" "$target" "$@")
where $base_prefix
points to the mounted AppImage, i.e. $target = $base_prefix/bin/pythonX.Y
. This trick seems to work fine in most cases. But, if the -E
option is used, then it obviously fails since PYTHONHOME
is ignored. This case is detected by the AppRun wrapper script. When -E
is used, then the wrapper directly calls the Python executable inside the AppImage, as
"$target" "$@"
Thus, sys.executable
points to $target
. This is OK for native calls. However, it fails when running inside venv. Then, sys.path
is initialised according to $target
, instead of $origin
.
As stated initialy, I do not have any good solution for now. The hacks that I have implemented for venv have some unsatisfactory side effects (e.g. spoiling PYTHONHOME), and anyway they do not work at 100%. On the other side, I like the fact that sys.executable
points to the AppImage executable (or to a corresponding symbolic link), rather than to a temporary mount point.
@nihilox ,
I have updated Python AppImages over the WE. They should now behave properly w.r.t. venv. Therefore, I am closing this issue for now. But, please feel free to re-open it if you still observe malfunctioning.
Technically speaking, I now patch sys.executable
directly at Python runtime. This happens after getpath
has initialised sys.prefix
, etc., but before the execution of site.main
. The latter sets sys.path
according to the result of getpath
, and depending on venv
. With this new method, I don't need to trick Python anymore with the exec
command and using PYTHONHOME
. Thus, Python AppImage should now behave close to native Python. However, in order to do so, I had to slightly modify core Python.
Note that as a side effect, it also patches the issue with pip
not being initialised at venv
creation. Now it should work properly.
Which should always be
/tmp/demo/venv/lib/python3.10/site-packages
.