Kernel frequently forgets about library installed in editable mode

Vincent-Liagre-QB commented 1 year ago

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[ ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[ ] Tried basic troubleshooting (if a bug/error)
- [x] Restarted Spyder
- [ ] Reset preferences with spyder --reset
- [ ] Reinstalled the latest version of Anaconda
- [ ] Tried the other applicable steps from the Troubleshooting Guide
[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Kernel frequently forgets about library installed in editable mode when running all file at once (F5): it will work for a few executions and then I'll get the No module named <my_module> error

Note that this was always working in previous Spyder versions I had

What steps reproduce the problem?

pip install -e . <my_module>
Run some file (F5) calling a few times

What is the expected output? What do you see instead?

Code to run normally

Paste Traceback/Error Below (if applicable)


Traceback (most recent call last):

  File xxxxxx\venv\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File xxxx\my_module\some_file2.py:11
    from my_module.some_file import some_function

  File <frozen importlib._bootstrap>:991 in _find_and_load

  File <frozen importlib._bootstrap>:971 in _find_and_load_unlocked

  File <frozen importlib._bootstrap>:914 in _find_spec

  File <frozen importlib._bootstrap_external>:1342 in find_spec

  File <frozen importlib._bootstrap_external>:1308 in _get_spec

  File <frozen importlib._bootstrap_external>:1174 in __iter__

  File <frozen importlib._bootstrap_external>:1162 in _recalculate

  File <frozen importlib._bootstrap_external>:1158 in _get_parent_path

KeyError: 'my_module'

Versions

Spyder version: 5.4.1
Python version: 3.8.8
Qt version: unsure
PyQt version: unsure
Operating System name/version: Windows 10

Also working with a virtualenv 20.17.1

Dependencies

# Mandatory:
atomicwrites >=1.2.0                 :  1.4.1 (OK)
chardet >=2.0.0                      :  5.1.0 (OK)
cloudpickle >=0.5.0                  :  2.2.0 (OK)
cookiecutter >=1.6.0                 :  2.1.1 (OK)
diff_match_patch >=20181111          :  20200713 (OK)
intervaltree                         :  None (OK)
IPython >=7.31.1;<9.0.0              :  8.7.0 (OK)
jedi >=0.17.2;<0.19.0                :  0.18.2 (OK)
jellyfish >=0.7                      :  0.9.0 (OK)
jsonschema >=3.2.0                   :  4.17.3 (OK)
keyring >=17.0.0                     :  23.13.1 (OK)
nbconvert >=4.0                      :  7.2.7 (OK)
numpydoc >=0.6.0                     :  1.5.0 (OK)
paramiko >=2.4.0                     :  2.12.0 (OK)
parso >=0.7.0;<0.9.0                 :  0.8.3 (OK)
pexpect >=4.4.0                      :  4.8.0 (OK)
pickleshare >=0.4                    :  0.7.5 (OK)
psutil >=5.3                         :  5.9.4 (OK)
pygments >=2.0                       :  2.13.0 (OK)
pylint >=2.5.0;<3.0                  :  2.15.9 (OK)
pylint_venv >=2.1.1                  :  2.3.0 (OK)
pyls_spyder >=0.4.0                  :  0.4.0 (OK)
pylsp >=1.7.0;<1.8.0                 :  1.7.0 (OK)
pylsp_black >=1.2.0                  :  1.2.1 (OK)
qdarkstyle >=3.0.2;<3.1.0            :  3.0.3 (OK)
qstylizer >=0.2.2                    :  0.2.2 (OK)
qtawesome >=1.2.1                    :  1.2.2 (OK)
qtconsole >=5.4.0;<5.5.0             :  5.4.0 (OK)
qtpy >=2.1.0                         :  2.3.0 (OK)
rtree >=0.9.7                        :  1.0.1 (OK)
setuptools >=49.6.0                  :  65.6.3 (OK)
sphinx >=0.6.6                       :  6.0.0 (OK)
spyder_kernels >=2.4.1;<2.5.0        :  2.4.1 (OK)
textdistance >=4.2.0                 :  4.5.0 (OK)
three_merge >=0.1.1                  :  0.1.1 (OK)
watchdog                             :  2.2.0 (OK)
zmq >=22.1.0                         :  24.0.1 (OK)

# Optional:
cython >=0.21                        :  0.29.32 (OK)
matplotlib >=3.0.0                   :  3.6.2 (OK)
numpy >=1.7                          :  1.24.1 (OK)
pandas >=1.1.1                       :  1.5.2 (OK)
scipy >=0.17.0                       :  1.9.3 (OK)
sympy >=0.7.3                        :  1.11.1 (OK)

# Spyder plugins:
spyder_terminal.terminalplugin 1.2.2 :  1.2.2 (OK)

dalthviz commented 1 year ago

Hi @Vincent-Liagre-QB thank you for the feedback! Could you try updating to Spyder 5.4.2 and check again? Also, could you share the example library you are installing in editable mode to try to reproduce this in our side?

Any new info to help us better understand what is happening here is greatly appreciated!

Vincent-Liagre-QB commented 1 year ago

Hi @dalthviz ; unfortunately upgrading didn't solve the issue ; will try and provide a reproducible example

ccordoba12 commented 1 year ago

@Vincent-Liagre-QB, please run the following command in the console, before running your code:

%autoreload off

That could help to solve your problem.

Vincent-Liagre-QB commented 1 year ago

Yes but then it will fail to reload my libs, right ? especially annoying for my lib in editable mode. Alternatively I found that CTRL-A + F9 is a good alternative to F5

ccordoba12 commented 1 year ago

Yes but then it will fail to reload my libs, right ?

Sure, but in that case you don't need to install them in editable mode. %autoreload simply reloads any library after you import it with from foo import bar. So, please try uninstall your modules to see if %autoreload can do its job without problems.

Alternatively I found that CTRL-A + F9 is a good alternative to F5

If you're satisfied with that solution, can we close this issue then? I mean, there's probably an bug between %autoreload and editable modules, but that's not something we can solve here. It needs to be reported and solved in IPython itself.

spyder-ide / spyder