search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.22k stars 1.59k forks source link

Segmentation fault when trying to reload files in network filesystems #21877

Open benoitbenoit opened 6 months ago

benoitbenoit commented 6 months ago

Description

What steps will reproduce the problem?

seg fault appears roughly 10 times a day during file edition. No idea why. I am using ubuntu 20.04. I am unable to find an issue. Thank you if you can.

Traceback

Fatal Python error: Segmentation fault

Thread 0x00007f8bc67fc700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/socket.py", line 293 in accept
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/spyder/app/mainwindow.py", line 1643 in start_open_files_server
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 917 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8bc6ffd700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 316 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 581 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/jupyter_client/channels.py", line 130 in _async_run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/events.py", line 80 in _run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 1905 in _run_once
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 634 in run_until_complete
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/jupyter_client/channels.py", line 149 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8c10f24700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/selectors.py", line 469 in select
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/asyncio/base_events.py", line 634 in run_until_complete
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/jupyter_client/threaded.py", line 266 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8c137fe700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 312 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 581 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/IPython/core/history.py", line 894 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/decorator.py", line 232 in fun
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8be4ff9700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 316 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/queue.py", line 180 in get
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/api.py", line 364 in dispatch_events
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/api.py", line 199 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8be57fa700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 312 in wait
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/utils/delayed_queue.py", line 54 in get
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/inotify_buffer.py", line 43 in read_event
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/inotify.py", line 129 in queue_events
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/api.py", line 148 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007f8be6ffd700 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/inotify_c.py", line 285 in read_events
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/watchdog/observers/inotify_buffer.py", line 88 in run
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/threading.py", line 937 in _bootstrap

Current thread 0x00007f8cb362a4c0 (most recent call first):
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/spyder/app/utils.py", line 333 in create_window
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/spyder/app/mainwindow.py", line 1819 in main
  File "/home/benoit/software/anaconda3/envs/spyder/lib/python3.9/site-packages/spyder/app/start.py", line 256 in main
  File "/home/benoit/software/anaconda3/envs/spyder/bin/spyder", line 11 in <module>

Versions

Dependencies

# Mandatory:
atomicwrites >=1.2.0                             :  1.4.0 (OK)
chardet >=2.0.0                                  :  4.0.0 (OK)
cloudpickle >=0.5.0                              :  2.2.1 (OK)
cookiecutter >=1.6.0                             :  2.5.0 (OK)
diff_match_patch >=20181111                      :  20200713 (OK)
intervaltree >=3.0.2                             :  3.1.0 (OK)
IPython >=7.31.1,<9.0.0,!=8.8.0,!=8.9.0,!=8.10.0 :  8.15.0 (OK)
jedi >=0.17.2,<0.19.0                            :  0.18.1 (OK)
jellyfish >=0.7                                  :  1.0.1 (OK)
jsonschema >=3.2.0                               :  4.19.2 (OK)
keyring >=17.0.0                                 :  23.13.1 (OK)
nbconvert >=4.0                                  :  7.10.0 (OK)
numpydoc >=0.6.0                                 :  1.5.0 (OK)
parso >=0.7.0,<0.9.0                             :  0.8.3 (OK)
pexpect >=4.4.0                                  :  4.8.0 (OK)
pickleshare >=0.4                                :  0.7.5 (OK)
psutil >=5.3                                     :  5.9.0 (OK)
pygments >=2.0                                   :  2.15.1 (OK)
pylint >=2.5.0,<3.0                              :  2.16.2 (OK)
pylint_venv >=2.1.1                              :  2.3.0 (OK)
pyls_spyder >=0.4.0                              :  0.4.0 (OK)
pylsp >=1.7.2,<1.8.0                             :  1.7.2 (OK)
pylsp_black >=1.2.0                              :  1.2.1 (OK)
qdarkstyle >=3.0.2,<3.2.0                        :  3.0.2 (OK)
qstylizer >=0.2.2                                :  0.2.2 (OK)
qtawesome >=1.2.1                                :  1.2.2 (OK)
qtconsole >=5.4.2,<5.5.0                         :  5.4.2 (OK)
qtpy >=2.1.0                                     :  2.4.1 (OK)
rtree >=0.9.7                                    :  1.0.1 (OK)
setuptools >=49.6.0                              :  68.2.2 (OK)
sphinx >=0.6.6                                   :  5.0.2 (OK)
spyder_kernels >=2.4.3,<2.5.0                    :  2.4.4 (OK)
textdistance >=4.2.0                             :  4.2.1 (OK)
three_merge >=0.1.1                              :  0.1.1 (OK)
watchdog >=0.10.3                                :  2.1.6 (OK)
xdg >=0.26                                       :  0.27 (OK)
zmq >=22.1.0                                     :  25.1.2 (OK)

# Optional:
cython >=0.21                                    :  3.0.6 (OK)
matplotlib >=3.0.0                               :  3.8.0 (OK)
numpy >=1.7                                      :  1.26.3 (OK)
pandas >=1.1.1                                   :  2.1.4 (OK)
scipy >=0.17.0                                   :  1.11.4 (OK)
sympy >=0.7.3                                    :  1.12 (OK)
ccordoba12 commented 6 months ago

Hey @benoitbenoit, thanks for reporting. The Linux packages provided by Anaconda are not very good, so please try with the Conda-forge ones (a set of community-maintained packages) to see if they solve the problem for you.

For that, please open a terminal and run there

conda create -n spyder-env -c conda-forge spyder python=3.10
conda activate spyder-env
spyder

If you need to work with the other Python libraries besides the ones that come with Spyder (e.g. Pandas), you need to install them in that new environment as well. For that you need to run

conda activate spyder-env
conda install -c conda-forge pandas

Let us know if that works for you.

benoitbenoit commented 6 months ago

Thank you for your reply. However, it did not solve the issue. I still have segfault few times a day.

Note that just before crashing, a pop-up window tells me that the file has been modified elsewhere and asks to recover it. I click cancel and then it crashes (sometimes it doesn't crash, it is really random). This is weird since the file is only opened in Spyder. Could this be the cause of the bug ?

Thank you again, Best

ccordoba12 commented 6 months ago

Thanks for the extra info @benoitbenoit. Is your home mounted in an NFS filesystem or something similar?

benoitbenoit commented 6 months ago

Thank you for your response, You are right, my scripts are stored on a server mounted with CIFS. Could this be source of the problem ? Thank you,

ccordoba12 commented 6 months ago

Yeah, I think it could be. We try to detect if a file has changed by reading its timestamp on disk, which I guess is handled differently by network filesystems than regular ones.

The only thing I can think of to fix this is to give users an option to disable that check instead of enforcing it automatically. But that is not a simple task, so I'm afraid we'll have to leave it for 6.1.0, to be released during the second half of the year. And there's no simple workaround to avoid it in the meantime, sorry.

benoitbenoit commented 6 months ago

Thank you. Best

and1bm commented 3 months ago

Hi, we can confirm the issue on Debian KDE Clients, where the home directories are mounted using CIFS. It causes a lot of annoyance in our programming classes. A fix is strongly appreciated. Let us know if we can provide any help/testing Many thanks for spyder and all your work!

and1bm commented 3 months ago

@benoitbenoit we give this patch a try:

+--- /usr/lib/python3/dist-packages/spyder/plugins/editor/widgets/editor.py     2024-06-20 07:16:54.096395325 +0200
++++ /usr/lib/python3/dist-packages/spyder/plugins/editor/widgets/editor.py     2024-06-20 14:39:07.693577124 +0200
+@@ -2370,15 +2370,16 @@
+         else:
+             # Else, testing if it has been modified elsewhere:
+             lastm = QFileInfo(finfo.filename).lastModified()
+-            if to_text_string(lastm.toString()) \
+-               != to_text_string(finfo.lastmodified.toString()):
++            dt = finfo.lastmodified.msecsTo(lastm)
++            if dt > 1000:
+                 if finfo.editor.document().isModified():
+                     self.msgbox = QMessageBox(
+                         QMessageBox.Question,
+                         self.title,
+-                        _("<b>%s</b> has been modified outside Spyder."
++                        _("It looks like <b>%s</b> has been modified "
++                          "outside Spyder. The working copy is from %i milliseconds ago."
+                           "<br>Do you want to reload it and lose all "
+-                          "your changes?") % name,
++                          "your changes?") % (name, dt),
+                         QMessageBox.Yes | QMessageBox.No,
+                         self)
+                     answer = self.msgbox.exec_()
ccordoba12 commented 3 months ago

@and1bm, thanks for the patch. Could you submit a pull request to give you feedback about it?

If you have time for that, don't forget to read our Contributing guide before doing it.

and1bm commented 3 months ago

Hi @ccordoba12 , not sure if this is more than a workaround. However, we will see if the problem described in this issue is fixed for us next week. Current master seems to have switched to another editor anyway.

ccordoba12 commented 2 months ago

Ok, please let us know if it works. I already left a review in your pull request if that's true.

benoitbenoit commented 2 months ago

Many thanks, this edit solved the issue.

I still sometimes get this error message: _The file xxxxx.py has been modified outside Spyder. Do you want to reload it and lose all your changes? Yes/No But Spyder no longer crashes when I select 'No'

Thank you again