Spyder 6.0a1 - Pandas error crashes console

[mg3146]

Hi - quick bug. It seems like when you get a pandas error, like this one, it kills the Spyder iPython console:

[mg3146]

Here is a reproducible example.

import pandas as pd

data = range(5)
index = range(3)
cols = range(4)

df = pd.DataFrame(data = data, columns = cols, index = index) 

If there was a way to fix this one for the next release, I would really appreciate it. As its killing my productivity (I still seem to have issues with 5), and I love using Spyder

[ccordoba12]

Hey @mg3146, thanks for reporting. Are you getting these errors when using Spyder 5?

[mg3146]

I can't even get Spyder 5 to launch anymore, so I've just been using 6. This wasn't a problem for me in Spyder 5 though when I was using it. Are you not able to reproduce?

[ccordoba12]

Not on Linux. @dalthviz, can you reproduce this problem on Windows?

[mg3146]

maybe its something to do with this corrupted environment of mine that's causing issues with both this and Spyder 5. If it helps, when I try to launch Spyder 5 from the anaconda powershell prompt I get the following traceback before it crashes:

[ccordoba12]

If you have Spyder 5 and 6 in the same env, I think that should be the cause. Please create a new env for Spyder 6 and report back if you're still having this problem.

[mg3146]

Took a look into that. Spyder 6 I used your standalone installer. For the console, I had originally been using an environment that Spyder 5 installed in it (via conda install spyder). I created a new environment without Spyder 5 installed, and I'm still getting the same crash / bug.

Here is a conda list of the environment I'm using for the console.

``` aiofiles 22.1.0 py39haa95532_0 aiohttp 3.8.4 pypi_0 pypi aiosignal 1.2.0 pyhd3eb1b0_0 aiosqlite 0.18.0 py39haa95532_0 anyio 3.5.0 py39haa95532_0 appdirs 1.4.4 pyhd3eb1b0_0 argon2-cffi 21.3.0 pyhd3eb1b0_0 argon2-cffi-bindings 21.2.0 py39h2bbff1b_0 arrow 1.2.3 pypi_0 pypi astroid 2.14.2 py39haa95532_0 asttokens 2.0.5 pyhd3eb1b0_0 async-timeout 4.0.2 py39haa95532_0 attrs 23.1.0 pypi_0 pypi autopep8 1.6.0 pyhd3eb1b0_1 babel 2.11.0 py39haa95532_0 backcall 0.2.0 pyhd3eb1b0_0 bcrypt 3.2.0 py39h2bbff1b_1 beanie 1.18.0 pypi_0 pypi beautifulsoup4 4.12.0 py39haa95532_0 bidict 0.21.2 pyhd3eb1b0_0 black 23.3.0 py39haa95532_0 blas 1.0 mkl bleach 4.1.0 pyhd3eb1b0_0 blpapi 3.19.3 pypi_0 pypi boost-cpp 1.73.0 h2bbff1b_12 boto3 1.26.137 pypi_0 pypi botocore 1.29.137 pypi_0 pypi bottleneck 1.3.5 py39h080aedc_0 brotli 1.0.9 h2bbff1b_7 brotli-bin 1.0.9 h2bbff1b_7 brotlipy 0.7.0 py39h2bbff1b_1003 bs4 4.12.0 py38hd3eb1b0_0 ca-certificates 2023.05.30 haa95532_0 cachetools 5.3.0 pypi_0 pypi cairo 1.16.0 haedb8bc_5 certifi 2023.5.7 py39haa95532_0 cffi 1.15.1 py39h2bbff1b_3 charset-normalizer 3.1.0 pypi_0 pypi click 8.1.3 pypi_0 pypi cloudpickle 2.2.1 py39haa95532_0 colorama 0.4.6 py39haa95532_0 comm 0.1.2 py39haa95532_0 contourpy 1.0.7 pypi_0 pypi convertdate 2.3.2 pyhd3eb1b0_0 cryptography 41.0.1 pypi_0 pypi cycler 0.11.0 pyhd3eb1b0_0 debugpy 1.5.1 py39hd77b12b_0 decorator 5.1.1 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 dill 0.3.6 py39haa95532_0 docstring-to-markdown 0.11 py39haa95532_0 docutils 0.18.1 py39haa95532_3 entrypoints 0.4 py39haa95532_0 exceptiongroup 1.1.1 pypi_0 pypi executing 0.8.3 pyhd3eb1b0_0 expat 2.4.9 h6c2663c_0 flake8 6.0.0 py39haa95532_0 font-ttf-dejavu-sans-mono 2.37 hd3eb1b0_0 font-ttf-inconsolata 2.001 hcb22688_0 font-ttf-source-code-pro 2.030 hd3eb1b0_0 font-ttf-ubuntu 0.83 h8b1ccd4_0 fontconfig 2.14.1 h9c4af85_2 fonts-anaconda 1 h8fa9717_0 fonts-conda-ecosystem 1 hd3eb1b0_0 fonttools 4.25.0 pyhd3eb1b0_0 fqdn 1.5.1 pypi_0 pypi freetype 2.12.1 ha860e81_0 fribidi 1.0.10 h62dcd97_0 frozenlist 1.3.3 py39h2bbff1b_0 getopt-win32 0.1 h2bbff1b_0 giflib 5.2.1 h8cc25b3_3 gitdb 4.0.7 pyhd3eb1b0_0 gitpython 3.1.30 py39haa95532_0 glib 2.69.1 h5dc1a3c_2 gprof2dot 2022.7.29 pypi_0 pypi graphite2 1.3.14 hd77b12b_1 graphviz 2.50.0 h7eca76f_1 greenlet 2.0.1 py39hd77b12b_0 grpcio 1.53.0 pypi_0 pypi gst-plugins-base 1.18.5 h9e645db_0 gstreamer 1.18.5 hd78058f_0 gts 0.7.6 h63ab5a1_3 harfbuzz 4.3.0 hb646838_1 hijri-converter 2.2.2 pyhd3eb1b0_0 holidays 0.18 py39haa95532_0 icc_rt 2022.1.0 h6049295_2 icu 58.2 ha925a31_3 idna 3.4 py39haa95532_0 importlib-metadata 6.0.0 py39haa95532_0 importlib_metadata 6.0.0 hd3eb1b0_0 importlib_resources 5.2.0 pyhd3eb1b0_1 iniconfig 2.0.0 pypi_0 pypi intel-openmp 2021.4.0 haa95532_3556 ipykernel 6.19.2 py39hd4e2768_0 ipynb-py-convert 0.4.6 pyh9f0ad1d_0 conda-forge ipython 8.14.0 pypi_0 pypi ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 8.0.4 py39haa95532_0 isoduration 20.11.0 pypi_0 pypi isort 5.9.3 pyhd3eb1b0_0 jedi 0.18.1 py39haa95532_1 jinja2 3.1.2 py39haa95532_0 jmespath 0.10.0 pyhd3eb1b0_0 jpeg 9e h2bbff1b_1 json5 0.9.6 pyhd3eb1b0_0 jsonpointer 2.4 pypi_0 pypi jsonschema 4.18.2 pypi_0 pypi jsonschema-specifications 2023.6.1 pypi_0 pypi jupyter 1.0.0 py39haa95532_8 jupyter_client 8.1.0 py39haa95532_0 jupyter_console 6.6.3 py39haa95532_0 jupyter_core 5.3.0 py39haa95532_0 jupyter_events 0.6.3 py39haa95532_0 jupyter_server 2.5.0 py39haa95532_0 jupyter_server_fileid 0.9.0 py39haa95532_0 jupyter_server_terminals 0.4.4 py39haa95532_1 jupyter_server_ydoc 0.8.0 py39haa95532_1 jupyter_ydoc 0.2.4 py39haa95532_0 jupyterlab 3.6.3 py39haa95532_0 jupyterlab_pygments 0.1.2 py_0 jupyterlab_server 2.22.0 py39haa95532_0 jupyterlab_widgets 3.0.5 py39haa95532_0 kaleido 0.2.1 pypi_0 pypi kiwisolver 1.4.4 py39hd77b12b_0 korean_lunar_calendar 0.2.1 pyhd3eb1b0_0 krb5 1.19.4 h5b6d351_0 lazy-model 0.0.5 pypi_0 pypi lazy-object-proxy 1.6.0 py39h2bbff1b_0 lcms2 2.12 h83e58a3_0 lerc 3.0 hd77b12b_0 libboost 1.73.0 h6c2663c_12 libbrotlicommon 1.0.9 h2bbff1b_7 libbrotlidec 1.0.9 h2bbff1b_7 libbrotlienc 1.0.9 h2bbff1b_7 libclang 14.0.6 default_hb5a9fac_1 libclang13 14.0.6 default_h8e68704_1 libcurl 8.1.1 h86230a5_1 libdeflate 1.17 h2bbff1b_0 libffi 3.4.4 hd77b12b_0 libgd 2.3.3 hd77b12b_2 libiconv 1.16 h2bbff1b_2 libogg 1.3.5 h2bbff1b_1 libpng 1.6.39 h8cc25b3_0 libsodium 1.0.18 h62dcd97_0 libssh2 1.10.0 hcd4344a_2 libtiff 4.5.0 h6c2663c_2 libvorbis 1.3.7 he774522_0 libwebp 1.2.4 hbc33d0d_1 libwebp-base 1.2.4 h2bbff1b_1 libxml2 2.10.3 h0ad7f3c_0 libxslt 1.1.37 h2bbff1b_0 lxml 4.9.2 py39h2bbff1b_0 lz4-c 1.9.4 h2bbff1b_0 mako 1.2.3 py39haa95532_0 markupsafe 2.1.1 py39h2bbff1b_0 matplotlib 3.7.1 py39haa95532_1 matplotlib-base 3.7.1 py39hf11a4ad_1 matplotlib-inline 0.1.6 py39haa95532_0 mccabe 0.7.0 pyhd3eb1b0_0 mistune 0.8.4 py39h2bbff1b_1000 mkl 2021.4.0 haa95532_640 mkl-service 2.4.0 py39h2bbff1b_0 mkl_fft 1.3.1 py39h277e83a_0 mkl_random 1.2.2 py39hf11a4ad_0 motor 3.1.2 pypi_0 pypi multidict 6.0.2 py39h2bbff1b_0 munkres 1.1.4 py_0 mypy_extensions 0.4.3 py39haa95532_1 nbclassic 0.5.5 py39haa95532_0 nbclient 0.5.13 py39haa95532_0 nbconvert 6.5.4 py39haa95532_0 nbformat 5.7.0 py39haa95532_0 nest-asyncio 1.5.6 py39haa95532_0 networkx 2.8.4 py39haa95532_1 notebook 6.5.3 py39haa95532_0 notebook-shim 0.2.2 py39haa95532_0 numexpr 2.8.4 py39h5b0cc5e_0 numpy 1.23.5 py39h3b20f71_0 numpy-base 1.23.5 py39h4da318b_0 openai 0.27.4 pypi_0 pypi openjpeg 2.4.0 h4fc8c34_0 openssl 1.1.1u h2bbff1b_0 packaging 23.0 py39haa95532_0 pandas 1.5.3 py39hf11a4ad_0 pandocfilters 1.5.0 pyhd3eb1b0_0 pango 1.50.7 h78c2152_0 paramiko 2.11.0 pypi_0 pypi parso 0.8.3 pyhd3eb1b0_0 pathspec 0.10.3 py39haa95532_0 pcre 8.45 hd77b12b_0 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 9.4.0 py39hd77b12b_0 pip 23.1.2 py39haa95532_0 pixman 0.40.0 h2bbff1b_1 platformdirs 2.5.2 py39haa95532_0 pluggy 1.0.0 py39haa95532_1 ply 3.11 py39haa95532_0 pooch 1.4.0 pyhd3eb1b0_0 poppler 22.12.0 h268424c_0 poppler-data 0.4.11 haa95532_1 powershell_shortcut 0.0.1 3 prometheus_client 0.14.1 py39haa95532_0 prompt-toolkit 3.0.36 py39haa95532_0 prompt_toolkit 3.0.36 hd3eb1b0_0 protobuf 4.22.1 pypi_0 pypi psutil 5.9.0 py39h2bbff1b_0 pure_eval 0.2.2 pyhd3eb1b0_0 pycodestyle 2.10.0 py39haa95532_0 pycountry 22.3.5 pypi_0 pypi pycparser 2.21 pyhd3eb1b0_0 pydantic 1.10.7 pypi_0 pypi pydocstyle 6.3.0 py39haa95532_0 pydot 1.4.1 py39haa95532_0 pyflakes 3.0.1 py39haa95532_0 pygments 2.15.1 py39haa95532_1 pylint 2.16.2 py39haa95532_0 pymeeus 0.5.11 pyhd3eb1b0_1 pynacl 1.5.0 py39h8cc25b3_0 pyopenssl 23.0.0 py39haa95532_0 pyparsing 3.0.9 py39haa95532_0 pyqt 5.15.7 py39hd77b12b_0 pyqt5-sip 12.11.0 py39hd77b12b_0 pyrsistent 0.18.0 py39h196d8e1_0 pysftp 0.2.9 pyhd3eb1b0_1 pyside6 6.5.0 pypi_0 pypi pyside6-addons 6.5.0 pypi_0 pypi pyside6-essentials 6.5.0 pypi_0 pypi pysocks 1.7.1 py39haa95532_0 pytest 7.2.2 pypi_0 pypi python 3.9.16 h6244533_2 python-dateutil 2.8.2 pyhd3eb1b0_0 python-fastjsonschema 2.16.2 py39haa95532_0 python-json-logger 2.0.7 py39haa95532_0 python-lsp-black 1.2.1 py39haa95532_0 python-lsp-jsonrpc 1.0.0 pyhd3eb1b0_0 python-lsp-server 1.7.1 py39haa95532_0 pytoolconfig 1.2.5 py39haa95532_1 pytz 2022.7 py39haa95532_0 pywin32 305 py39h2bbff1b_0 pywinpty 2.0.10 py39h5da7b33_0 pyxll 5.6.4 pypi_0 pypi pyxll-jupyter 0.4.1 pypi_0 pypi pyyaml 6.0 py39h2bbff1b_1 pyzmq 25.1.0 py39hd77b12b_0 qt-main 5.15.2 he8e5bd7_8 qt-webengine 5.15.9 hb9a9bb5_5 qtconsole 5.4.2 py39haa95532_0 qtpy 2.2.0 py39haa95532_0 qtwebkit 5.212 h2bbfb41_5 referencing 0.29.1 pypi_0 pypi requests 2.29.0 py39haa95532_0 rfc3339-validator 0.1.4 py39haa95532_0 rfc3986-validator 0.1.1 py39haa95532_0 rope 1.7.0 py39haa95532_0 rpds-py 0.8.10 pypi_0 pypi s3transfer 0.6.0 py39haa95532_0 scipy 1.10.1 py39h321e85e_0 send2trash 1.8.0 pyhd3eb1b0_1 setuptools 67.8.0 py39haa95532_0 shiboken6 6.5.0 pypi_0 pypi sip 6.6.2 py39hd77b12b_0 six 1.16.0 pyhd3eb1b0_1 smmap 4.0.0 pyhd3eb1b0_0 snakeviz 2.1.1 pypi_0 pypi sniffio 1.2.0 py39haa95532_1 snowballstemmer 2.2.0 pyhd3eb1b0_0 soupsieve 2.4 py39haa95532_0 spyder-kernels 3.0.0b1 pypi_0 pypi sqlalchemy 1.4.39 py39h2bbff1b_0 sqlite 3.41.2 h2bbff1b_0 stack_data 0.2.0 pyhd3eb1b0_0 tenacity 8.0.1 py39haa95532_1 terminado 0.17.1 py39haa95532_0 texttable 1.6.4 pyhd3eb1b0_0 tinycss2 1.2.1 py39haa95532_0 tk 8.6.12 h2bbff1b_0 toml 0.10.2 pyhd3eb1b0_0 tomli 2.0.1 py39haa95532_0 tomlkit 0.11.1 py39haa95532_0 tornado 6.2 py39h2bbff1b_0 tqdm 4.65.0 pypi_0 pypi traitlets 5.7.1 py39haa95532_0 typing-extensions 4.6.3 py39haa95532_0 typing_extensions 4.6.3 py39haa95532_0 tzdata 2023c h04d1e81_0 ujson 5.4.0 py39hd77b12b_0 uri-template 1.3.0 pypi_0 pypi urllib3 1.26.15 py39haa95532_0 vc 14.2 h21ff451_1 vf-core 2.5 pypi_0 pypi vs2015_runtime 14.27.29016 h5e58377_2 wcwidth 0.2.5 pyhd3eb1b0_0 webcolors 1.13 pypi_0 pypi webencodings 0.5.1 py39haa95532_1 websocket-client 0.58.0 py39haa95532_4 whatthepatch 1.0.2 py39haa95532_0 wheel 0.38.4 py39haa95532_0 widgetsnbextension 4.0.5 py39haa95532_0 win_inet_pton 1.1.0 py39haa95532_0 winpty 0.4.3 4 wrapt 1.14.1 py39h2bbff1b_0 xmltodict 0.12.0 pyhd3eb1b0_0 xz 5.4.2 h8cc25b3_0 y-py 0.5.9 py39hb6bf4ef_0 yaml 0.2.5 he774522_0 yapf 0.31.0 pyhd3eb1b0_0 yarl 1.8.1 py39h2bbff1b_0 ypy-websocket 0.8.2 py39haa95532_0 zeromq 4.3.4 hd77b12b_0 zipp 3.11.0 py39haa95532_0 zlib 1.2.13 h8cc25b3_0 zstd 1.5.5 hd43e919_0 ```

[ccordoba12]

No worries, I tested on Windows and I was able to reproduce this problem with the code you provided above. We'll try to fix it for alpha2.

[ccordoba12]

@impact27, this is a very serious bug and it doesn't happen on 5.x, which means it should be related to the many changes you did to the way we handle kernels.

Could you take a look at it? Thanks!

[impact27]

This is a tricky one. For some reason inspecting a frame while a pandas object is being created crashes python. This also means that debugging this would crash python. The problem here is that when a traceback is created in Spyder 6, the frames are inspected to provide helpful information about the crash. We could remove this but the underlying issue, which is probably a pandas issue, would still be there.

[impact27]

Can you check if https://github.com/spyder-ide/spyder-kernels/pull/464 fixes this issue?

[impact27]

To reproduce, from a terminal:

% ipython
Python 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd

In [2]: data = range(5)
   ...: index = range(3)
   ...: cols = range(4)

In [3]: %debug pd.DataFrame(data = data, columns = cols, index = index)
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> <string>(1)<module>()

ipdb> s
--Call--
> /opt/homebrew/lib/python3.10/site-packages/pandas/core/frame.py(608)__init__()
    606     # Constructors
    607 
--> 608     def __init__(
    609         self,
    610         data=None,

ipdb> self.anythingatall
*** RecursionError: maximum recursion depth exceeded while calling a Python object
ipdb> 

On macOS, I get a "RecursionError", on windows, python crashes.

[impact27]

Should I open a bug in pandas? It is not clear this is really an issue unless you are debugging, in which case you are interfering with the program flow. I think we should probably not get the traceback locals as this might get triggered.

[ccordoba12]

Should I open a bug in pandas?

Yeah, please do.

It is not clear this is really an issue unless you are debugging, in which case you are interfering with the program flow. I think we should probably not get the traceback locals as this might get triggered.

I saw that you already did that, but I agree.