mu-editor / mu

A small, simple editor for beginner Python programmers. Written in Python and Qt5.
http://codewith.mu
GNU General Public License v3.0
1.41k stars 435 forks source link

Issue launching virtualenv module isolated from user's environment #1954

Open carlosperate opened 2 years ago

carlosperate commented 2 years ago

This is a follow up from the solution to:

Which was to add the -I flag to the python -m virtualenv ... command: https://github.com/mu-editor/mu/pull/1921/files

$ python --help
...
-I     : isolate Python from the user's environment (implies -E and -s)
...

This is perfect for the installer Mu instances, as everything needed is bundled and we want to isolated from any user Python environment. However, it introduces an issue in some Linux distros, or at least Debian based ones, and maybe others too (and maybe even Windows, as it has the option to install packages in %APPDATA%\Python, but needs to be tested).

In these Python environments pip install by default installs packages in the "user's environment" and so, they are not present to the interpreter when using the -I flag.

So when launching Mu and it tries to create a virtualenv with the -I flag it cannot find the module:

Traceback (most recent call last):
  File "/home/parallels/.local/lib/python3.8/site-packages/mu/app.py", line 159, in run
    raise ex
  File "/home/parallels/.local/lib/python3.8/site-packages/mu/app.py", line 146, in run
    venv.ensure_and_create(self.display_text)
  File "/home/parallels/.local/lib/python3.8/site-packages/mu/virtual_environment.py", line 603, in ensure_and_create
    self.create()
  File "/home/parallels/.local/lib/python3.8/site-packages/mu/virtual_environment.py", line 771, in create
    self.create_venv()
  File "/home/parallels/.local/lib/python3.8/site-packages/mu/virtual_environment.py", line 803, in create_venv
    raise VirtualEnvironmentCreateError(
mu.virtual_environment.VirtualEnvironmentCreateError: Unable to create a virtual environment using /usr/bin/python3 at /home/parallels/.local/share/mu/mu_venv-38-20211226-233553
STDERR: /usr/bin/python3: No module named virtualenv

To show this in a relatively clean Ubuntu 20.04, after running python3 -m pip install mu-editor==1.1.0b7, these are the packages locations:

$ python3 -m pip list --verbose
Package                           Version              Location                                           Installer
--------------------------------- -------------------- -------------------------------------------------- ---------
adafruit-board-toolkit            1.1.0                /home/parallels/.local/lib/python3.8/site-packages pip
appdirs                           1.4.4                /home/parallels/.local/lib/python3.8/site-packages pip
apturl                            0.5.2                /usr/lib/python3/dist-packages
backcall                          0.2.0                /home/parallels/.local/lib/python3.8/site-packages pip
backports.entry-points-selectable 1.1.1                /home/parallels/.local/lib/python3.8/site-packages pip
bcrypt                            3.1.7                /usr/lib/python3/dist-packages
black                             21.12b0              /home/parallels/.local/lib/python3.8/site-packages pip
blinker                           1.4                  /usr/lib/python3/dist-packages
Brlapi                            0.7.0                /usr/lib/python3/dist-packages
certifi                           2019.11.28           /usr/lib/python3/dist-packages
chardet                           3.0.4                /usr/lib/python3/dist-packages
click                             8.0.3                /home/parallels/.local/lib/python3.8/site-packages pip
colorama                          0.4.3                /usr/lib/python3/dist-packages
command-not-found                 0.3                  /usr/lib/python3/dist-packages
cryptography                      2.8                  /usr/lib/python3/dist-packages
cupshelpers                       1.0                  /usr/lib/python3/dist-packages
dbus-python                       1.2.16               /usr/lib/python3/dist-packages
decorator                         5.1.0                /home/parallels/.local/lib/python3.8/site-packages pip
defer                             1.0.6                /usr/lib/python3/dist-packages
distlib                           0.3.4                /home/parallels/.local/lib/python3.8/site-packages pip
distro                            1.4.0                /usr/lib/python3/dist-packages
distro-info                       0.23ubuntu1          /usr/lib/python3/dist-packages
duplicity                         0.8.12.0             /usr/lib/python3/dist-packages
entrypoints                       0.3                  /usr/lib/python3/dist-packages
fasteners                         0.14.1               /usr/lib/python3/dist-packages
filelock                          3.4.2                /home/parallels/.local/lib/python3.8/site-packages pip
flake8                            4.0.1                /home/parallels/.local/lib/python3.8/site-packages pip
future                            0.18.2               /usr/lib/python3/dist-packages
httplib2                          0.14.0               /usr/lib/python3/dist-packages
idna                              2.8                  /usr/lib/python3/dist-packages
ipykernel                         5.5.6                /home/parallels/.local/lib/python3.8/site-packages pip
ipython                           7.30.1               /home/parallels/.local/lib/python3.8/site-packages pip
ipython-genutils                  0.2.0                /home/parallels/.local/lib/python3.8/site-packages pip
jedi                              0.18.1               /home/parallels/.local/lib/python3.8/site-packages pip
jupyter-client                    6.1.12               /home/parallels/.local/lib/python3.8/site-packages pip
jupyter-core                      4.9.1                /home/parallels/.local/lib/python3.8/site-packages pip
keyring                           18.0.1               /usr/lib/python3/dist-packages
language-selector                 0.1                  /usr/lib/python3/dist-packages
launchpadlib                      1.10.13              /usr/lib/python3/dist-packages
lazr.restfulclient                0.14.2               /usr/lib/python3/dist-packages
lazr.uri                          1.0.3                /usr/lib/python3/dist-packages
lockfile                          0.12.2               /usr/lib/python3/dist-packages
louis                             3.12.0               /usr/lib/python3/dist-packages
macaroonbakery                    1.3.1                /usr/lib/python3/dist-packages
Mako                              1.1.0                /usr/lib/python3/dist-packages
MarkupSafe                        1.1.0                /usr/lib/python3/dist-packages
matplotlib-inline                 0.1.3                /home/parallels/.local/lib/python3.8/site-packages pip
mccabe                            0.6.1                /home/parallels/.local/lib/python3.8/site-packages pip
monotonic                         1.5                  /usr/lib/python3/dist-packages
mu-editor                         1.1.0b7              /home/parallels/.local/lib/python3.8/site-packages pip
mypy-extensions                   0.4.3                /home/parallels/.local/lib/python3.8/site-packages pip
netifaces                         0.10.4               /usr/lib/python3/dist-packages
nudatus                           0.0.5                /home/parallels/.local/lib/python3.8/site-packages pip
oauthlib                          3.1.0                /usr/lib/python3/dist-packages
olefile                           0.46                 /usr/lib/python3/dist-packages
packaging                         21.3                 /home/parallels/.local/lib/python3.8/site-packages pip
paramiko                          2.6.0                /usr/lib/python3/dist-packages
parso                             0.8.3                /home/parallels/.local/lib/python3.8/site-packages pip
pathspec                          0.9.0                /home/parallels/.local/lib/python3.8/site-packages pip
pexpect                           4.6.0                /usr/lib/python3/dist-packages
pickleshare                       0.7.5                /home/parallels/.local/lib/python3.8/site-packages pip
Pillow                            7.0.0                /usr/lib/python3/dist-packages
pip                               21.3.1               /home/parallels/.local/lib/python3.8/site-packages pip
platformdirs                      2.4.1                /home/parallels/.local/lib/python3.8/site-packages pip
prompt-toolkit                    3.0.24               /home/parallels/.local/lib/python3.8/site-packages pip
protobuf                          3.6.1                /usr/lib/python3/dist-packages
pycairo                           1.16.2               /usr/lib/python3/dist-packages
pycodestyle                       2.8.0                /home/parallels/.local/lib/python3.8/site-packages pip
pycups                            1.9.73               /usr/lib/python3/dist-packages
pyflakes                          2.4.0                /home/parallels/.local/lib/python3.8/site-packages pip
Pygments                          2.10.0               /home/parallels/.local/lib/python3.8/site-packages pip
PyGObject                         3.36.0               /usr/lib/python3/dist-packages
PyJWT                             1.7.1                /usr/lib/python3/dist-packages
pymacaroons                       0.13.0               /usr/lib/python3/dist-packages
PyNaCl                            1.3.0                /usr/lib/python3/dist-packages
pyparsing                         3.0.6                /home/parallels/.local/lib/python3.8/site-packages pip
PyQt5                             5.13.2               /home/parallels/.local/lib/python3.8/site-packages pip
PyQt5-sip                         12.9.0               /home/parallels/.local/lib/python3.8/site-packages pip
PyQtChart                         5.13.1               /home/parallels/.local/lib/python3.8/site-packages pip
pyRFC3339                         1.1                  /usr/lib/python3/dist-packages
pyserial                          3.5                  /home/parallels/.local/lib/python3.8/site-packages pip
python-apt                        2.0.0+ubuntu0.20.4.1 /usr/lib/python3/dist-packages
python-dateutil                   2.7.3                /usr/lib/python3/dist-packages
python-debian                     0.1.36ubuntu1        /usr/lib/python3/dist-packages
pytz                              2019.3               /usr/lib/python3/dist-packages
pyxdg                             0.26                 /usr/lib/python3/dist-packages
PyYAML                            5.3.1                /usr/lib/python3/dist-packages
pyzmq                             22.3.0               /home/parallels/.local/lib/python3.8/site-packages pip
QScintilla                        2.11.3               /home/parallels/.local/lib/python3.8/site-packages pip
qtconsole                         4.7.7                /home/parallels/.local/lib/python3.8/site-packages pip
QtPy                              2.0.0                /home/parallels/.local/lib/python3.8/site-packages pip
reportlab                         3.5.34               /usr/lib/python3/dist-packages
requests                          2.22.0               /usr/lib/python3/dist-packages
requests-unixsocket               0.2.0                /usr/lib/python3/dist-packages
SecretStorage                     2.3.1                /usr/lib/python3/dist-packages
semver                            2.13.0               /home/parallels/.local/lib/python3.8/site-packages pip
setuptools                        45.2.0               /usr/lib/python3/dist-packages
simplejson                        3.16.0               /usr/lib/python3/dist-packages
six                               1.14.0               /usr/lib/python3/dist-packages
systemd-python                    234                  /usr/lib/python3/dist-packages
tomli                             1.2.3                /home/parallels/.local/lib/python3.8/site-packages pip
tornado                           6.1                  /home/parallels/.local/lib/python3.8/site-packages pip
traitlets                         5.1.1                /home/parallels/.local/lib/python3.8/site-packages pip
typing_extensions                 4.0.1                /home/parallels/.local/lib/python3.8/site-packages pip
ubuntu-advantage-tools            20.3                 /usr/lib/python3/dist-packages
ubuntu-drivers-common             0.0.0                /usr/lib/python3/dist-packages
ufw                               0.36                 /usr/lib/python3/dist-packages
unattended-upgrades               0.1                  /usr/lib/python3/dist-packages
urllib3                           1.25.8               /usr/lib/python3/dist-packages
usb-creator                       0.3.7                /usr/lib/python3/dist-packages
virtualenv                        20.10.0              /home/parallels/.local/lib/python3.8/site-packages pip
wadllib                           1.3.3                /usr/lib/python3/dist-packages
wcwidth                           0.2.5                /home/parallels/.local/lib/python3.8/site-packages pip
wheel                             0.34.2               /usr/lib/python3/dist-packages
xkit                              0.0.0                /usr/lib/python3/dist-packages

Basically anything from the OS is installed in /usr/lib/python3/dist-packages and everything installed by the user from the pip install mu-editor command is in /home/parallels/.local/lib/python3.8/site-packages.

Using the -I flag we can see that the interpreter doesn't see any of the Mu dependencies:

$ python3 -I -m pip list --verbose
Package                Version              Location                       Installer
---------------------- -------------------- ------------------------------ ---------
apturl                 0.5.2                /usr/lib/python3/dist-packages          
bcrypt                 3.1.7                /usr/lib/python3/dist-packages          
blinker                1.4                  /usr/lib/python3/dist-packages          
Brlapi                 0.7.0                /usr/lib/python3/dist-packages          
certifi                2019.11.28           /usr/lib/python3/dist-packages          
chardet                3.0.4                /usr/lib/python3/dist-packages          
Click                  7.0                  /usr/lib/python3/dist-packages          
colorama               0.4.3                /usr/lib/python3/dist-packages          
command-not-found      0.3                  /usr/lib/python3/dist-packages          
cryptography           2.8                  /usr/lib/python3/dist-packages          
cupshelpers            1.0                  /usr/lib/python3/dist-packages          
dbus-python            1.2.16               /usr/lib/python3/dist-packages          
defer                  1.0.6                /usr/lib/python3/dist-packages          
distro                 1.4.0                /usr/lib/python3/dist-packages          
distro-info            0.23ubuntu1          /usr/lib/python3/dist-packages          
duplicity              0.8.12.0             /usr/lib/python3/dist-packages          
entrypoints            0.3                  /usr/lib/python3/dist-packages          
fasteners              0.14.1               /usr/lib/python3/dist-packages          
future                 0.18.2               /usr/lib/python3/dist-packages          
httplib2               0.14.0               /usr/lib/python3/dist-packages          
idna                   2.8                  /usr/lib/python3/dist-packages          
keyring                18.0.1               /usr/lib/python3/dist-packages          
language-selector      0.1                  /usr/lib/python3/dist-packages          
launchpadlib           1.10.13              /usr/lib/python3/dist-packages          
lazr.restfulclient     0.14.2               /usr/lib/python3/dist-packages          
lazr.uri               1.0.3                /usr/lib/python3/dist-packages          
lockfile               0.12.2               /usr/lib/python3/dist-packages          
louis                  3.12.0               /usr/lib/python3/dist-packages          
macaroonbakery         1.3.1                /usr/lib/python3/dist-packages          
Mako                   1.1.0                /usr/lib/python3/dist-packages          
MarkupSafe             1.1.0                /usr/lib/python3/dist-packages          
monotonic              1.5                  /usr/lib/python3/dist-packages          
netifaces              0.10.4               /usr/lib/python3/dist-packages          
oauthlib               3.1.0                /usr/lib/python3/dist-packages          
olefile                0.46                 /usr/lib/python3/dist-packages          
paramiko               2.6.0                /usr/lib/python3/dist-packages          
pexpect                4.6.0                /usr/lib/python3/dist-packages          
Pillow                 7.0.0                /usr/lib/python3/dist-packages          
pip                    20.0.2               /usr/lib/python3/dist-packages          
protobuf               3.6.1                /usr/lib/python3/dist-packages          
pycairo                1.16.2               /usr/lib/python3/dist-packages          
pycups                 1.9.73               /usr/lib/python3/dist-packages          
PyGObject              3.36.0               /usr/lib/python3/dist-packages          
PyJWT                  1.7.1                /usr/lib/python3/dist-packages          
pymacaroons            0.13.0               /usr/lib/python3/dist-packages          
PyNaCl                 1.3.0                /usr/lib/python3/dist-packages          
pyRFC3339              1.1                  /usr/lib/python3/dist-packages          
python-apt             2.0.0+ubuntu0.20.4.1 /usr/lib/python3/dist-packages          
python-dateutil        2.7.3                /usr/lib/python3/dist-packages          
python-debian          0.1.36ubuntu1        /usr/lib/python3/dist-packages          
pytz                   2019.3               /usr/lib/python3/dist-packages          
pyxdg                  0.26                 /usr/lib/python3/dist-packages          
PyYAML                 5.3.1                /usr/lib/python3/dist-packages          
reportlab              3.5.34               /usr/lib/python3/dist-packages          
requests               2.22.0               /usr/lib/python3/dist-packages          
requests-unixsocket    0.2.0                /usr/lib/python3/dist-packages          
SecretStorage          2.3.1                /usr/lib/python3/dist-packages          
setuptools             45.2.0               /usr/lib/python3/dist-packages          
simplejson             3.16.0               /usr/lib/python3/dist-packages          
six                    1.14.0               /usr/lib/python3/dist-packages          
systemd-python         234                  /usr/lib/python3/dist-packages          
ubuntu-advantage-tools 20.3                 /usr/lib/python3/dist-packages          
ubuntu-drivers-common  0.0.0                /usr/lib/python3/dist-packages          
ufw                    0.36                 /usr/lib/python3/dist-packages          
unattended-upgrades    0.1                  /usr/lib/python3/dist-packages          
urllib3                1.25.8               /usr/lib/python3/dist-packages          
usb-creator            0.3.7                /usr/lib/python3/dist-packages          
wadllib                1.3.3                /usr/lib/python3/dist-packages          
wheel                  0.34.2               /usr/lib/python3/dist-packages          
xkit                   0.0.0                /usr/lib/python3/dist-packages   

Crash reports:

carlosperate commented 2 years ago

Possible solutions:

Current workarounds for Linux users:

tmontes commented 2 years ago

@carlosperate, just released pup 1.0.0a15 that defaults to launching with the -I flag.

rickcox commented 2 years ago

SUGGESTION - You would see alot less newbies (like Me!) running into this if you were to update the Mu install instructions at: https://codewith.mu/en/howto/1.1/install_with_python to recommend installing in a virtual environment. Ref: (Dup Issue mu-editor/mu#2144)

carlosperate commented 2 years ago

Thanks for the feedback @rickcox!

Technically speaking, the instructions already indicate this:

image

But it is not very visible and I agree that having this emphasised would likely help users.