search

microsoft / vscode-jupyter

VS Code Jupyter extension
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter
MIT License
1.26k stars 276 forks source link

Kernel crash when view dataframe with period index #10446

Closed DereckZhang closed 2 years ago

DereckZhang commented 2 years ago

Applies To

What happened?

I got a simple dataframe with period index, and it can be calculated correctly. But click show variable in data viewer in jupyter-variables panel, kernel crashed and notification like: image Then, I remove its period index, and now it can be viewed properly.

VS Code Version

Version: 1.68.0 Commit: 4af164ea3a06f701fe3e89a2bcbb421d2026b68f Date: 2022-06-08T11:44:16.822Z Electron: 17.4.7 Chromium: 98.0.4758.141 Node.js: 16.13.0 V8: 9.8.177.13-electron.0 OS: Windows_NT x64 10.0.19044

Jupyter Extension Version

v2022.5.1001601848

Jupyter logs

Visual Studio Code (1.68.0, undefined, desktop)
Jupyter Extension Version: 2022.5.1001601848.
Python Extension Version: 2022.8.0.
Workspace folder d:\CJ\FoF\Quantamental
info 22:41:57.508: ZMQ install verified.
User belongs to experiment group 'jupyterTestcf'
User belongs to experiment group 'jupyterEnhancedDataViewer'
info 22:41:57.731: LSP Notebooks experiment is disabled -- not in treatment group
info 22:41:57.742: Experiment status for python is {"enabled":true,"optInto":[],"optOutFrom":[]}
info 22:41:58.347: Registering dummy command feature
info 22:41:58.787: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:58.788: Find preferred kernel for d:\CJ\FoF\Quantamental\backtest.ipynb with metadata {"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":2},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython2","version":"2.7.6"}} & preferred interpreter d:\env\condaenv\conda310\python.exe
info 22:41:58.790: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:58.790: Preferred kernel .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher is exact match
info 22:41:58.791: PreferredConnection: .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher found for NotebookDocument: d:\CJ\FoF\Quantamental\backtest.ipynb
info 22:41:58.793: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:58.803: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:58.803: Find preferred kernel for d:\CJ\FoF\Quantamental\backtest.ipynb with metadata {"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":2},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython2","version":"2.7.6"}} & preferred interpreter d:\env\condaenv\conda310\python.exe
info 22:41:58.804: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:58.804: Preferred kernel .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher is exact match
info 22:41:58.989: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:59.11: Starting Jupyter Session id = 'startUsingPythonInterpreter:.jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher' (Python Path: d:\env\condaenv\conda310, EnvType: Conda, EnvName: '', Version: 3.10.4) for 'd:\CJ\FoF\Quantamental\backtest.ipynb' (disableUI=true)
info 22:41:59.11: Creating raw notebook for resource 'd:\CJ\FoF\Quantamental\backtest.ipynb'
info 22:41:59.11: Computing working directory for resource 'd:\CJ\FoF\Quantamental\backtest.ipynb'
info 22:41:59.32: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:59.32: Find preferred kernel for d:\CJ\FoF\Quantamental\backtest.ipynb with metadata {"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":2},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython2","version":"2.7.6"}} & preferred interpreter d:\env\condaenv\conda310\python.exe
info 22:41:59.34: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:59.34: Preferred kernel .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher is exact match
info 22:41:59.35: PreferredConnection: .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher found for NotebookDocument: d:\CJ\FoF\Quantamental\backtest.ipynb
info 22:41:59.41: Starting raw kernel 'conda310 (Python 3.10.4)' for interpreter d:\env\condaenv\conda310\python.exe
info 22:41:59.89: Kernel launching with ports 9000,9001,9002,9003,9004. Start port is 9000
info 22:41:59.112: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:59.112: Find preferred kernel for d:\CJ\FoF\Quantamental\backtest.ipynb with metadata {"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":2},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython2","version":"2.7.6"}} & preferred interpreter d:\env\condaenv\conda310\python.exe
info 22:41:59.114: Preferred Remote kernel for d:\CJ\FoF\Quantamental\backtest.ipynb is undefined
info 22:41:59.114: Preferred kernel .jvsc74a57bd0348ebc624cc7119089e2fe4cfbb9de781be47776c85de75c084bfda3b3053470.d:\env\condaenv\conda310\python.exe.d:\env\condaenv\conda310\python.exe.-m#ipykernel_launcher is exact match
info 22:41:59.131: Process Execution: > d:\env\condaenv\conda310\python.exe -m pip list
> d:\env\condaenv\conda310\python.exe -m pip list
info 22:41:59.135: Process Execution: > d:\env\condaenv\conda310\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\env\condaenv\conda310\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 22:41:59.157: Creating daemon process for d:\env\condaenv\conda310\python.exe with env variables count 69
info 22:41:59.161: Process Execution: > d:\env\condaenv\conda310\python.exe -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_interrupt_daemon -v --ppid 21192
> d:\env\condaenv\conda310\python.exe -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_interrupt_daemon -v --ppid 21192
info 22:41:59.388: get interrupthandle daemon
info 22:41:59.394: Process Execution: > d:\env\condaenv\conda310\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"a34263af-3cb7-430a-902f-2c27d7ca0072" --shell=9002 --transport="tcp" --iopub=9004 --f=d:\Users\Dereck\AppData\Roaming\jupyter\runtime\kernel-v2-21192kJjq0HBWvHuR.json
> d:\env\condaenv\conda310\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"a34263af-3cb7-430a-902f-2c27d7ca0072" --shell=9002 --transport="tcp" --iopub=9004 --f=d:\Users\Dereck\AppData\Roaming\jupyter\runtime\kernel-v2-21192kJjq0HBWvHuR.json
info 22:41:59.394: Process Execution: cwd: d:\CJ\FoF\Quantamental
cwd: d:\CJ\FoF\Quantamental
info 22:41:59.606: ipykernel version 6.13.0 for d:\env\condaenv\conda310\python.exe
info 22:41:59.606: ipykernel location d:\env\condaenv\conda310\lib\site-packages\ipykernel\__init__.py for d:\env\condaenv\conda310\python.exe
warn 22:42:0.837: StdErr from Kernel Process d:\env\condaenv\conda310\lib\site-packages\traitlets\traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(

warn 22:42:0.837: StdErr from Kernel Process d:\env\condaenv\conda310\lib\site-packages\traitlets\traitlets.py:2346: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use 'a34263af-3cb7-430a-902f-2c27d7ca0072' instead of 'b"a34263af-3cb7-430a-902f-2c27d7ca0072"'.
  warn(

info 22:42:0.905: Kernel Output: NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

To read more about this, see https://github.com/ipython/ipython/issues/2049

To connect another client to this kernel, use:
    --existing d:\Users\Dereck\AppData\Roaming\jupyter\runtime\kernel-v2-21192kJjq0HBWvHuR.json

info 22:42:1.91: Started kernel conda310 (Python 3.10.4), (Raw session started and connected)
info 22:42:1.92: Finished connecting 1fd869b8-9563-4db5-b5b9-9d308f93b45b
info 22:42:1.149: UpdateWorkingDirectoryAndPath in Kernel
info 22:42:1.149: Add SVG to matplotlib figure formats
info 22:42:1.149: Executing silently Code (idle) = import os\nimport sys\n%cd "d:\CJ\FoF\Quantamental"\nif os.getcwd() not in sys.path:\nsys.path.inser
info 22:42:1.955: Executing silently Code (completed) = import os\nimport sys\n%cd "d:\CJ\FoF\Quantamental"\nif os.getcwd() not in sys.path:\nsys.path.inser
info 22:42:1.955: Waiting for idle on (kernel): af4a319e-1d1d-47aa-be21-c5ef972588f2 -> idle
info 22:42:1.955: Finished waiting for idle on (kernel): af4a319e-1d1d-47aa-be21-c5ef972588f2 -> idle
info 22:42:3.403: Execute Cell 0 d:\CJ\FoF\Quantamental\backtest.ipynb
info 22:42:3.755: Cell 0 executed with state Success
info 22:42:8.826: Execute Cell 2 d:\CJ\FoF\Quantamental\backtest.ipynb
info 22:42:8.903: Cell 2 executed with state Success
info 22:42:17.535: Loading webview. View is notset
info 22:42:17.536: Loading web view...
info 22:42:17.536: Webview panel created.
info 22:42:17.736: Web view react rendered
info 22:42:17.776: Executing silently Code (idle) = # Query Jupyter server for the info about a dataframe\nimport json as _VSCODE_json\nimport builtins as
info 22:42:17.779: Executing silently Code (idle) = # Query Jupyter server for the info about a dataframe\nimport json as _VSCODE_json\nimport builtins as
info 22:42:17.798: Executing silently Code (completed) = # Query Jupyter server for the info about a dataframe\nimport json as _VSCODE_json\nimport builtins as
info 22:42:17.799: Executing silently Code (idle) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
info 22:42:17.814: Executing silently Code (completed) = # Query Jupyter server for the info about a dataframe\nimport json as _VSCODE_json\nimport builtins as
info 22:42:17.815: Executing silently Code (idle) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
info 22:42:17.830: Executing silently Code (completed) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
info 22:42:17.848: Executing silently Code (completed) = import builtins\n_rwho_ls = %who_ls\nbuiltins.print(_VSCODE_getVariableTypes(_rwho_ls))
info 22:42:17.852: Executing silently Code (idle) = import builtins\nbuiltins.print(_VSCODE_getVariableInfo(bm))
info 22:42:17.856: Executing silently Code (idle) = import builtins\nbuiltins.print(_VSCODE_getVariableInfo(bm))
info 22:42:17.862: Executing silently Code (completed) = import builtins\nbuiltins.print(_VSCODE_getVariableInfo(bm))
info 22:42:17.878: Executing silently Code (completed) = import builtins\nbuiltins.print(_VSCODE_getVariableInfo(bm))
info 22:42:20.473: Loading webview. View is notset
info 22:42:20.474: Loading web view...
info 22:42:20.474: Webview panel created.
info 22:42:20.550: Process Execution: > d:\env\condaenv\conda310\python.exe -c "import pandas;print(pandas.__version__)"
> d:\env\condaenv\conda310\python.exe -c "import pandas;print(pandas.__version__)"
info 22:42:20.733: Web view react rendered
info 22:42:21.56: Executing silently Code (idle) = import pandas as _VSCODE_pd\nimport builtins as _VSCODE_builtins\nimport json as _VSCODE_json\nimport n
info 22:42:21.85: Executing silently Code (completed) = import pandas as _VSCODE_pd\nimport builtins as _VSCODE_builtins\nimport json as _VSCODE_json\nimport n
info 22:42:21.85: Executing silently Code (idle) = import builtins\nbuiltins.print(_VSCODE_getDataFrameInfo(pick))
error 22:42:21.107: Disposing session as kernel process died ExitCode: 3221225477, Reason: d:\env\condaenv\conda310\lib\site-packages\traitlets\traitlets.py:2392: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(
d:\env\condaenv\conda310\lib\site-packages\traitlets\traitlets.py:2346: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use 'a34263af-3cb7-430a-902f-2c27d7ca0072' instead of 'b"a34263af-3cb7-430a-902f-2c27d7ca0072"'.
  warn(

info 22:42:21.108: kill daemon
error 22:42:21.109: Raw kernel process exited code: 3221225477
error 22:42:21.112: [Error: Canceled future for execute_request message before replies were done
    at t.KernelShellFutureHandler.dispose (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:1376085)
    at d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:1395137
    at Map.forEach (<anonymous>)
    at y._clearKernelState (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:1395122)
    at y.dispose (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:1388604)
    at d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:578260
    at t.swallowExceptions (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:1001797)
    at dispose (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:578238)
    at t.RawSession.dispose (d:\Users\Dereck\.vscode\extensions\ms-toolsai.jupyter-2022.5.1001601848\out\extension.node.js:2:583202)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)]

Coding Language and Runtime Version

Python v.3.10.4

Language Extension Version (if applicable)

v2022.8.0

Anaconda Version (if applicable)

miniconda 4.13.0

Running Jupyter locally or remotely?

Local

rchiodo commented 2 years ago

Thanks for the bug, do you have the example code that causes the issue?

rchiodo commented 2 years ago

I tried this example I found. There was no crash however:

import pandas as pd

course = ["DBMS", "DSA", "OOPS",
          "System Design", "CN", ]

# pass the period and starting index
webinar_date = pd.period_range('2020-08-15', periods=5)

# Determine Period Index and Column
# for DataFrame
df = pd.DataFrame(course, index=webinar_date, columns=['Course'])
DereckZhang commented 2 years ago

@rchiodo I have tried your code and error repeats again. I slightly change your code as follow:

import pandas as pd

course = ["DBMS", "DSA", "OOPS",
          "System Design", "CN", ]

# pass the period and starting index
webinar_date = pd.period_range('2020-08-15', periods=5)

# Determine Period Index and Column
# for DataFrame to compare
df1 = pd.DataFrame(course, index=webinar_date, columns=['Course'])
df2 = pd.DataFrame(course, columns=['Course'])

df2 can be viewed properly, but df1 shows error as described. Is there any other info I could provide?

rchiodo commented 2 years ago

What is the version of python and pandas in use? I'm using 3.9.7 and pandas 1.3.4

rchiodo commented 2 years ago

Error code 3221225477 is an access violation so I'd guess pandas is crashing for some reason.

DereckZhang commented 2 years ago

@rchiodo You are right! I use conda to manager enviroment and set pandas=1.3.4(with python 3.9.7), then df1 can be viewed as: image It seems that index can't show properly.

rchiodo commented 2 years ago

What version of pandas and python were you using before?

PeriodIndex showing an object is expected at the moment. We really only support simple data types for viewing in a column.

DereckZhang commented 2 years ago

@rchiodo I used pandas 1.4.2 and python 3.10.4 before, and every env is mananged by conda separately.

rchiodo commented 2 years ago

Thanks I can repro the crash with those versions. Probably a bug in pandas.

rchiodo commented 2 years ago

Weird part is I tried doing the same thing by executing our dataframe parsing code in cells and it didn't crash.

IanMatthewHuff commented 2 years ago

Closing this down as an issue with pandas.