search

spyder-ide / spyder

Official repository for Spyder - The Scientific Python Development Environment
https://www.spyder-ide.org
MIT License
8.34k stars 1.62k forks source link

Timeout error when trying to copy a variable in the Variable Explorer #12913

Open DIV-on-github opened 4 years ago

DIV-on-github commented 4 years ago

Description

What steps will reproduce the problem?

I am parsing a file containing all sorts of weird characters, but treating it as if it were ANSI.
I have a somewhat complicated regex expression.
The regex search sometimes fails in two ways. (1) More commonly the generator from re.finditer seems to be successfully created, but when looping with a "for ... in ...:" structure after iterating through to the last genuine match, it then loops to a further fictious match and then hangs. That is when running the file normally or in the debugger.
(2) In this case I was debugging and the regex hung immediately after calling re.search (in an imported module).

Traceback

  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\ipythonconsole\widgets\namespacebrowser.py", line 96, in get_value
    return self.call_kernel(
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder_kernels\comms\commbase.py", line 537, in __call__
    return self._comms_wrapper._get_call_return_value(
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\ipythonconsole\comms\kernelcomm.py", line 131, in _get_call_return_value
    return super(KernelComm, self)._get_call_return_value(
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder_kernels\comms\commbase.py", line 415, in _get_call_return_value
    self._wait_reply(call_id, call_name, timeout)
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\ipythonconsole\comms\kernelcomm.py", line 142, in _wait_reply
    self._wait(got_reply, self._sig_got_reply, timeout_msg, timeout)
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\ipythonconsole\comms\kernelcomm.py", line 168, in _wait
    raise TimeoutError(timeout_msg)
TimeoutError: Timeout while waiting for {'3f415961d7bb40758874483d85d38ac0': (False, <bound method ShellWidget.remote_set_cwd of <spyder.plugins.ipythonconsole.widgets.shell.ShellWidget object at 0x0000017532028D30>>), '026462dd222441e1ae932997c06f3294': (False, <bound method ShellWidget.remote_set_cwd of <spyder.plugins.ipythonconsole.widgets.shell.ShellWidget object at 0x0000017532028D30>>), 'f2c5f495478f4d5e886b11d2cb6bcb74': (False, <bound method ShellWidget.remote_set_cwd of <spyder.plugins.ipythonconsole.widgets.shell.ShellWidget object at 0x0000017532028D30>>), '0994e966213d497fa58e22ed98a7301a': (False, <bound method ShellWidget.remote_set_cwd of <spyder.plugins.ipythonconsole.widgets.shell.ShellWidget object at 0x0000017532028D30>>), '43907024bcc943f7928a767ac677b835': (True, None), '86e38be36ddd48d0953122011fcacc12': (True, None), 'fed3f10fd78d43fc8b8a1695372f0bec': (True, None), 'c881786c090a4e35b13fe56af7672b64': (True, None), '83321a208c564b4292faee9e067ce13f': (True, None)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\variableexplorer\widgets\collectionseditor.py", line 1040, in copy
    obj = self.delegate.get_value(idx)
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\variableexplorer\widgets\collectionseditor.py", line 1383, in get_value
    return self.parent().get_value(name)
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\variableexplorer\widgets\collectionseditor.py", line 1438, in get_value
    value = self.shellwidget.get_value(name)
  File "C:\Program files (portable)\WinPy_3-8\WPy64-3830\python-3.8.3.amd64\lib\site-packages\spyder\plugins\ipythonconsole\widgets\namespacebrowser.py", line 101, in get_value
    raise ValueError(msg % reason_big)
ValueError: The variable is too big to be retrieved.<br><br>Note: Please don't report this problem on Github, there's nothing to do about it.

Versions

Dependencies


# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.4.1 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.14.0 (OK)
jedi >=0.15.2                  :  0.17.0 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  0.9.2 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso >=0.5.2                  :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.0 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=0.25                  :  2.5.2 (OK)
pyls >=0.31.9;<0.33.0          :  0.32.0 (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.7.2 (OK)
qtconsole >=4.6.0              :  4.7.4 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
sphinx >=0.6.6                 :  3.0.3 (OK)
spyder_kernels >=1.9.1;<1.10.0 :  1.9.1 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.1 (OK)

# Optional:
cython >=0.21                  :  0.29.19 (OK)
matplotlib >=2.0.0             :  3.2.1 (OK)
numpy >=1.7                    :  1.18.4 (OK)
pandas >=0.13.1                :  1.0.3 (OK)
scipy >=0.17.0                 :  1.4.1 (OK)
sympy >=0.7.3                  :  1.5.1 (OK)
DIV-on-github commented 4 years ago

I forgot to say that in the second instance while the debugger seemed to be stuck (as if processing an infinite loop or something) I went to the variables explorer pane and tried to copy a string variable's content, which was the path to a file.
After no immediate response, after 5 to 10 seconds I got several warning/error messages stating that the variable was too big to copy.

DIV-on-github commented 4 years ago

These are pretty annoying, because when they happen the only thing to do is close down the console or the kernel. The button to "Stop the current command" doesn't work at all.

DIV-on-github commented 4 years ago

I am stepping through the second issue, and the code hangs when it gets to the last line of def _compile(pattern, flags): which is return p

DIV-on-github commented 4 years ago

By the way, although the source text is someone long (about 15 MB), that is not the cause of the problem. The same code can be run successfully on files of similar content that are twice as large (although it might take a dozen seconds, it doesn't hang for the other files I read in).

goanpeca commented 4 years ago

@DIV-on-github thanks for the report.

Could you please share with us a minimal reproducible example of what you are doing step by step (with some example content) so that we can reproduce it on our side?

Thanks

DIV-on-github commented 4 years ago

I would like to, but it could be a large amount of work for me to find the responsible characters in the files that are read in, because: (1) often they can be analysed without error; and (2) previously every time the analysis failed Spyder would hang for an unlimited amount of time.
I finally managed to implement the wrapt_timeout_decorator https://github.com/bitranox/wrapt_timeout_decorator/blob/master/README.rst#use-with-windows so that at least I can get an exception after a specified duration.
If I can isolate it further I'll report back.