Compatibility with pysparkkernel (kernel may hang forever)

Environment data

VS Code version: 1.92.2 (Universal) fee1edb8d6d72a0ddff41e5f71a671c23ed924b9
Jupyter Extension version (available under the Extensions sidebar): v2024.7.0
Python Extension version (available under the Extensions sidebar): v2024.12.3
OS (Windows | Mac | Linux distro) and version: MacOS 14.3.1 (23D60)
Python and/or Anaconda version: Python 3.8.19
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
Jupyter server running: Local

Expected behaviour

pysparkkernel (sparkmagic) respond normally like other kernels

Actual behaviour

After starting pysparkkernel and executing the first cell, the kernel may hang for a long time. Even if the first cell responds after several minutes, the execution of second cell may hang forever.

I observe the kernel logs and notice that when vscode first execute a cell, it will send some initial code like import sys as _VSCODE_sys\nprint(_VSCODE_sys.executable); del _VSCODE_sys to kernel. However, in pysparkkernel, executing common python code equals to sending code to remote livy server. When there is no spark session exists, the pysparkkernel will start a spark session. Hence, the initial code is wrongly recognized as user-executed code, then pysparkkernel begins to start spark session, which explains why kernel hangs for several minutes (kernel sends execution reply until spark session is ready). After that, the communication between vscode-jupyter and kernel becomes very unstable, sometimes cells can be executed, sometimes it will hang forever (maybe some channel is in wrong state because of long initializing time or incorrect kernel reply?). In contrast, jupyter lab works fine with pysparkkernel because it does not send initial code to kernel.

I patched the pysparkkernel and let it execute initial code sent by vscode jupyter locally, then everything works fine.

I think this is a compatibility issue between vscode-jupyter and pysparkkernel, initial code should not be sent to pysparkkernel.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

Installing sparkmagic and pysparkkernel
Choose pysparkkernel in vscode (need to fill in pysparkkernel config, see sparkmagic project)
Execute code in vsocde.

Logs

Output for Jupyter in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Jupyter)

``` 23:16:09.233 [trace] Conda file is conda 23:16:11.109 [debug] Get Custom Env Variables, Class name = Sm, completed in 0ms, has a truthy return value, Arg 1: undefined, Arg 2: "RunPythonCode" 23:16:11.116 [debug] KernelProvider switched kernel to id = .undefined.sparkmagic.kernels.pysparkkernel.pysparkkernel./Users/~/miniforge3/envs/sparkmagictest/python./Users/~/miniforge3/envs/sparkmagictest/python.-m#sparkmagic.kernels.pysparkkernel.pysparkkernel 23:16:11.116 [debug] start the kernel, options.disableUI=true for ~/Desktop/livy-emr.ipynb 23:16:11.127 [trace] Registering commtarget jupyter.widget 23:16:11.128 [debug] Controller selection change completed 23:16:11.128 [info] Starting Kernel (Python Path: ~/miniforge3/envs/sparkmagictest/bin/python, Conda, 3.8.19) for '~/Desktop/livy-emr.ipynb' (disableUI=true) 23:16:11.128 [trace] Creating raw notebook for resource '~/Desktop/livy-emr.ipynb' 23:16:11.131 [debug] Get Custom Env Variables, Class name = Sm, completed in 1ms, has a truthy return value, Arg 1: "~/Desktop/livy-emr.ipynb", Arg 2: "RunPythonCode" 23:16:11.138 [debug] Launching kernel .undefined.sparkmagic.kernels.pysparkkernel.pysparkkernel./Users/~/miniforge3/envs/sparkmagictest/python./Users/~/miniforge3/envs/sparkmagictest/python.-m#sparkmagic.kernels.pysparkkernel.pysparkkernel for ~/Desktop/livy-emr.ipynb in ~/Desktop with ports 9001, 9000, 9004, 9002, 9003 23:16:11.138 [info] Launching Raw Kernel PySpark # /Users/~/miniforge3/envs/sparkmagictest/bin/python 23:16:11.138 [debug] Create ProcessService, Class name = Im (started execution), Arg 1: "~/Desktop/livy-emr.ipynb", Arg 2: {"a":false,"b":{"z":3,"w":{"i":-1,"end":0},"u":[{"id":2044},{"id":2045},null,{"id":2048}]}} 23:16:11.138 [debug] Get Custom Env Variables, Class name = Sm, completed in 0ms, has a truthy return value, Arg 1: "~/Desktop/livy-emr.ipynb", Arg 2: "RunNonPythonCode", Arg 3: {"a":false,"b":{"z":3,"w":{"i":-1,"end":0},"u":[{"id":2044},{"id":2045},null,{"id":2048}]}} 23:16:11.139 [debug] Create ProcessService, Class name = Im, completed in 1ms, has a truthy return value, Arg 1: "~/Desktop/livy-emr.ipynb", Arg 2: {"a":false,"b":{"z":3,"w":{"i":-1,"end":0},"u":[{"id":2044},{"id":2045},null,{"id":2048}]}} 23:16:11.968 [debug] Got env vars from Python Ext in 838ms for ~/miniforge3/envs/sparkmagictest/bin/python, with env var count 62. 23:16:11.968 [trace] Prepend PATH with python bin for ~/miniforge3/envs/sparkmagictest/bin/python 23:16:11.968 [debug] Getting activated env variables, Class name = pd, completed in 838ms, has a truthy return value, Arg 1: "~/Desktop/livy-emr.ipynb", Arg 2: "~/miniforge3/envs/sparkmagictest/bin/python", Arg 3: undefined 23:16:11.969 [info] Process Execution: ~/miniforge3/envs/sparkmagictest/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)" 23:16:11.970 [info] Process Execution: ~/miniforge3/envs/sparkmagictest/bin/python -m sparkmagic.kernels.pysparkkernel.pysparkkernel -f /Users/~/Library/Jupyter/runtime/kernel-v2-57879km7dCukFoby5.json > cwd: ~/Desktop 23:16:11.970 [debug] Kernel process 58066. 23:16:11.972 [debug] Got env vars from Python Ext in 863ms for ~/miniforge3/envs/sparkmagictest/bin/python, with env var count 62. 23:16:11.972 [trace] Prepend PATH with python bin for ~/miniforge3/envs/sparkmagictest/bin/python 23:16:11.972 [debug] Getting activated env variables, Class name = pd, completed in 863ms, has a truthy return value, Arg 1: undefined, Arg 2: "~/miniforge3/envs/sparkmagictest/bin/python", Arg 3: undefined 23:16:11.972 [info] Process Execution: ~/miniforge3/envs/sparkmagictest/bin/python -m pip list 23:16:12.208 [trace] ipykernel version & path 6.29.5, ~/miniforge3/envs/sparkmagictest/lib/python3.8/site-packages/ipykernel/__init__.py for ~/miniforge3/envs/sparkmagictest/bin/python 23:16:12.569 [debug] Handle Execution of Cells 0 for ~/Desktop/livy-emr.ipynb 23:16:13.570 [debug] Kernel output 58066: To connect another client to this kernel, use: --existing kernel-v2-57879km7dCukFoby5.json 23:16:13.599 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{},"header":{"date":"2024-08-31T15:16:13.599Z","msg_id":"c190b899-c532-49c4-86bf-bb637c48412e","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:16:13.599 [debug] Waiting for Raw Session to be ready in postStartRawSession 23:16:13.599 [debug] Waiting for Raw session to be ready, status: connected 23:16:13.599 [trace] Raw session connected 23:16:13.599 [debug] Waited for Raw session to be ready & got status: connected 23:16:13.600 [debug] Successfully waited for Raw Session to be ready in postStartRawSession 23:16:13.600 [debug] Kernel status is 'unknown' before requesting kernel info and after ready 23:16:13.600 [debug] Sending request for kernelInfo 23:16:13.600 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"control","content":{},"header":{"date":"2024-08-31T15:16:13.600Z","msg_id":"578714bf-8878-47e3-8081-19f5bc608e5a","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:16:13.600 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{},"header":{"date":"2024-08-31T15:16:13.600Z","msg_id":"2c0fedf7-7e50-4019-a630-451777bf34b6","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:16:13.779 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_0","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.778702Z","version":"5.3"},"parent_header":{},"metadata":{},"content":{"execution_state":"starting"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.781 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_1","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.780216Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.599000Z","msg_id":"c190b899-c532-49c4-86bf-bb637c48412e","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.781 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_3","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.780830Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"578714bf-8878-47e3-8081-19f5bc608e5a","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.781 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_5","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.781315Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.599000Z","msg_id":"c190b899-c532-49c4-86bf-bb637c48412e","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.782 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_6","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.781565Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"578714bf-8878-47e3-8081-19f5bc608e5a","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.782 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_2","msg_type":"kernel_info_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.780434Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.599000Z","msg_id":"c190b899-c532-49c4-86bf-bb637c48412e","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"status":"ok","protocol_version":"5.3","implementation":"PySpark","implementation_version":"1.0","language_info":{"name":"pyspark","mimetype":"text/x-python","codemirror_mode":{"name":"python","version":3},"file_extension":".py","pygments_lexer":"python3"},"banner":"Python 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:49:57) \nType 'copyright', 'credits' or 'license' for more information\nIPython 8.12.3 -- An enhanced Interactive Python. Type '?' for help.\n","help_links":[{"text":"Python Reference","url":"https://docs.python.org/3.8"},{"text":"IPython Reference","url":"https://ipython.org/documentation.html"},{"text":"NumPy Reference","url":"https://docs.scipy.org/doc/numpy/reference/"},{"text":"SciPy Reference","url":"https://docs.scipy.org/doc/scipy/reference/"},{"text":"Matplotlib Reference","url":"https://matplotlib.org/contents.html"},{"text":"SymPy Reference","url":"http://docs.sympy.org/latest/index.html"},{"text":"pandas Reference","url":"https://pandas.pydata.org/pandas-docs/stable/"}]},"idents":[],"buffers":[],"channel":"shell"} 23:16:13.782 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_4","msg_type":"kernel_info_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.781043Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"578714bf-8878-47e3-8081-19f5bc608e5a","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"status":"ok","protocol_version":"5.3","implementation":"PySpark","implementation_version":"1.0","language_info":{"name":"pyspark","mimetype":"text/x-python","codemirror_mode":{"name":"python","version":3},"file_extension":".py","pygments_lexer":"python3"},"banner":"Python 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:49:57) \nType 'copyright', 'credits' or 'license' for more information\nIPython 8.12.3 -- An enhanced Interactive Python. Type '?' for help.\n","help_links":[{"text":"Python Reference","url":"https://docs.python.org/3.8"},{"text":"IPython Reference","url":"https://ipython.org/documentation.html"},{"text":"NumPy Reference","url":"https://docs.scipy.org/doc/numpy/reference/"},{"text":"SciPy Reference","url":"https://docs.scipy.org/doc/scipy/reference/"},{"text":"Matplotlib Reference","url":"https://matplotlib.org/contents.html"},{"text":"SymPy Reference","url":"http://docs.sympy.org/latest/index.html"},{"text":"pandas Reference","url":"https://pandas.pydata.org/pandas-docs/stable/"}]},"idents":[],"buffers":[],"channel":"control"} 23:16:13.782 [trace] Got response for requestKernelInfo 23:16:13.782 [debug] Successfully completed postStartRawSession after 1 attempt(s) in 182ms 23:16:13.783 [trace] Executing silently Code (idle) = import sys as _VSCODE_sys\nprint(_VSCODE_sys.executable); del _VSCODE_sys 23:16:13.783 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"silent":false,"store_history":false,"user_expressions":{},"allow_stdin":true,"stop_on_error":false,"code":"import sys as _VSCODE_sys\nprint(_VSCODE_sys.executable); del _VSCODE_sys"},"header":{"date":"2024-08-31T15:16:13.783Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:16:13.783 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_7","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.783007Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"2c0fedf7-7e50-4019-a630-451777bf34b6","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.784 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_9","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.783932Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"2c0fedf7-7e50-4019-a630-451777bf34b6","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.784 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_8","msg_type":"kernel_info_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.783083Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.600000Z","msg_id":"2c0fedf7-7e50-4019-a630-451777bf34b6","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"status":"ok","protocol_version":"5.3","implementation":"PySpark","implementation_version":"1.0","language_info":{"name":"pyspark","mimetype":"text/x-python","codemirror_mode":{"name":"python","version":3},"file_extension":".py","pygments_lexer":"python3"},"banner":"Python 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:49:57) \nType 'copyright', 'credits' or 'license' for more information\nIPython 8.12.3 -- An enhanced Interactive Python. Type '?' for help.\n","help_links":[{"text":"Python Reference","url":"https://docs.python.org/3.8"},{"text":"IPython Reference","url":"https://ipython.org/documentation.html"},{"text":"NumPy Reference","url":"https://docs.scipy.org/doc/numpy/reference/"},{"text":"SciPy Reference","url":"https://docs.scipy.org/doc/scipy/reference/"},{"text":"Matplotlib Reference","url":"https://matplotlib.org/contents.html"},{"text":"SymPy Reference","url":"http://docs.sympy.org/latest/index.html"},{"text":"pandas Reference","url":"https://pandas.pydata.org/pandas-docs/stable/"}]},"idents":[],"buffers":[],"channel":"shell"} 23:16:13.785 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_10","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.785061Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:16:13.786 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_11","msg_type":"execute_input","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:13.785143Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"code":"import sys as _VSCODE_sys\nprint(_VSCODE_sys.executable); del _VSCODE_sys","execution_count":1},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,101,120,101,99,117,116,101,95,105,110,112,117,116]}],"buffers":[],"channel":"iopub"} 23:16:14.924 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_12","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:16:14.922838Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","date":"2024-08-31T15:16:13.783000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stdout","text":"Starting Spark application\n"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,111,117,116]}],"buffers":[],"channel":"iopub"} 23:17:15.984 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_13","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:15.982094Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"","text/html":"

ID	YARN Application ID	Kind	State	Spark UI	Driver log	User	Current session?
0	spark-f58b418b85ef4fccae42b8b768ce7d54	pyspark	idle	Link		None	✔

"},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"} 23:17:16.129 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_14","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:16.127631Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":"25px","justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"50%"},"buffer_paths":[]},"comm_id":"91c101c84433443cbda11a5e7f39f7c5","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,57,49,99,49,48,49,99,56,52,52,51,51,52,52,51,99,98,100,97,49,49,97,53,101,55,102,51,57,102,55,99,53]}],"buffers":[],"channel":"iopub"} 23:17:16.129 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_15","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:16.128196Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""},"buffer_paths":[]},"comm_id":"fa15423e60634a17b55eccaafafd451c","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,102,97,49,53,52,50,51,101,54,48,54,51,52,97,49,55,98,53,53,101,99,99,97,97,102,97,102,100,52,53,49,99]}],"buffers":[],"channel":"iopub"} 23:17:16.130 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_16","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:16.128295Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"info","description":"Progress:","description_allow_html":false,"layout":"IPY_MODEL_91c101c84433443cbda11a5e7f39f7c5","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_fa15423e60634a17b55eccaafafd451c","tabbable":null,"tooltip":null,"value":0},"buffer_paths":[]},"comm_id":"4a320c2c24fe4e438b1f5c71bb25e53d","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,52,97,51,50,48,99,50,99,50,52,102,101,52,101,52,51,56,98,49,102,53,99,55,49,98,98,50,53,101,53,51,100]}],"buffers":[],"channel":"iopub"} 23:17:16.130 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_17","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:16.129061Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"FloatProgress(value=0.0, bar_style='info', description='Progress:', layout=Layout(height='25px', width='50%'),…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4a320c2c24fe4e438b1f5c71bb25e53d"}},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"} 23:17:16.131 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"91c101c84433443cbda11a5e7f39f7c5","data":{}},"header":{"date":"2024-08-31T15:17:16.130Z","msg_id":"192e2c7b-3b35-4ef3-a205-af7c640b254d","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:16.131 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"fa15423e60634a17b55eccaafafd451c","data":{}},"header":{"date":"2024-08-31T15:17:16.131Z","msg_id":"4064053d-63e7-4249-8ffc-e63a33c95419","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:16.131 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"4a320c2c24fe4e438b1f5c71bb25e53d","data":{}},"header":{"date":"2024-08-31T15:17:16.131Z","msg_id":"87a59b21-669e-4edb-aefb-7eeaa226eac4","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:18.585 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_18","msg_type":"comm_close","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.584706Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{},"comm_id":"4a320c2c24fe4e438b1f5c71bb25e53d"},"idents":[{"type":"Buffer","data":[99,111,109,109,45,52,97,51,50,48,99,50,99,50,52,102,101,52,101,52,51,56,98,49,102,53,99,55,49,98,98,50,53,101,53,51,100]}],"buffers":[],"channel":"iopub"} 23:17:18.586 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_19","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.585795Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","date":"2024-08-31T15:16:13.783000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stdout","text":"SparkSession available as 'spark'.\n"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,111,117,116]}],"buffers":[],"channel":"iopub"} 23:17:18.693 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_20","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.691338Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":"25px","justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"50%"},"buffer_paths":[]},"comm_id":"978d9f30c1504d65b2fc334894a1bff9","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,57,55,56,100,57,102,51,48,99,49,53,48,52,100,54,53,98,50,102,99,51,51,52,56,57,52,97,49,98,102,102,57]}],"buffers":[],"channel":"iopub"} 23:17:18.694 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_21","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.692253Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""},"buffer_paths":[]},"comm_id":"978741196212441fa81a2227247d16a7","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,57,55,56,55,52,49,49,57,54,50,49,50,52,52,49,102,97,56,49,97,50,50,50,55,50,52,55,100,49,54,97,55]}],"buffers":[],"channel":"iopub"} 23:17:18.694 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_22","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.692510Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"info","description":"Progress:","description_allow_html":false,"layout":"IPY_MODEL_978d9f30c1504d65b2fc334894a1bff9","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_978741196212441fa81a2227247d16a7","tabbable":null,"tooltip":null,"value":0},"buffer_paths":[]},"comm_id":"e3eda4b210ae4e6b9044bbe00d8c4075","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,101,51,101,100,97,52,98,50,49,48,97,101,52,101,54,98,57,48,52,52,98,98,101,48,48,100,56,99,52,48,55,53]}],"buffers":[],"channel":"iopub"} 23:17:18.694 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"978d9f30c1504d65b2fc334894a1bff9","data":{}},"header":{"date":"2024-08-31T15:17:18.694Z","msg_id":"5efa94b6-39bb-4b99-9162-4848cbe51868","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:18.695 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"978741196212441fa81a2227247d16a7","data":{}},"header":{"date":"2024-08-31T15:17:18.695Z","msg_id":"d928486e-5e27-442e-ac98-47f981a3d461","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:18.695 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"comm_id":"e3eda4b210ae4e6b9044bbe00d8c4075","data":{}},"header":{"date":"2024-08-31T15:17:18.695Z","msg_id":"779acb1a-3010-4fe4-85de-3351b3fa995b","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:18.695 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_23","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.693854Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"FloatProgress(value=0.0, bar_style='info', description='Progress:', layout=Layout(height='25px', width='50%'),…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e3eda4b210ae4e6b9044bbe00d8c4075"}},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"} 23:17:18.739 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_24","msg_type":"comm_close","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.738350Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{},"comm_id":"e3eda4b210ae4e6b9044bbe00d8c4075"},"idents":[{"type":"Buffer","data":[99,111,109,109,45,101,51,101,100,97,52,98,50,49,48,97,101,52,101,54,98,57,48,52,52,98,98,101,48,48,100,56,99,52,48,55,53]}],"buffers":[],"channel":"iopub"} 23:17:18.740 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_25","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.739004Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","date":"2024-08-31T15:16:13.783000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stdout","text":"/usr/bin/python3"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,111,117,116]}],"buffers":[],"channel":"iopub"} 23:17:18.741 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_27","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.741155Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.741 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_26","msg_type":"execute_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.740739Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:16:13.783000Z","msg_id":"5478cd0a-8efd-44f7-b5d0-0c2ff28f81b3","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"started":"2024-08-31T15:16:13.785124Z","dependencies_met":true,"engine":"44ca996b-4012-4d50-8721-633ecb536a60","status":"ok"},"content":{"status":"ok","execution_count":0,"user_expressions":{},"payload":[]},"idents":[],"buffers":[],"channel":"shell"} 23:17:18.742 [trace] Executing silently Code (completed) = import sys as _VSCODE_sys\nprint(_VSCODE_sys.executable); del _VSCODE_sys with 6 output(s) 23:17:18.742 [trace] Started running kernel initialization for ~/Desktop/livy-emr.ipynb 23:17:18.742 [trace] Executing silently Code (idle) = try:\nimport ipywidgets as _VSCODE_ipywidgets\nprint("e976ee50-99ed-4aba-9b6b-9dcd5634d07d:IPy 23:17:18.743 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"silent":false,"store_history":false,"user_expressions":{},"allow_stdin":true,"stop_on_error":false,"code":"try:\n import ipywidgets as _VSCODE_ipywidgets\n print(\"e976ee50-99ed-4aba-9b6b-9dcd5634d07d:IPyWidgets:\" + _VSCODE_ipywidgets.__version__)\n del _VSCODE_ipywidgets\nexcept:\n pass"},"header":{"date":"2024-08-31T15:17:18.743Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}} 23:17:18.743 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_28","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.742484Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.130000Z","msg_id":"192e2c7b-3b35-4ef3-a205-af7c640b254d","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.744 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_29","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.743770Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.130000Z","msg_id":"192e2c7b-3b35-4ef3-a205-af7c640b254d","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.745 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_30","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.744301Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.131000Z","msg_id":"4064053d-63e7-4249-8ffc-e63a33c95419","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.745 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_31","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.745342Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.131000Z","msg_id":"4064053d-63e7-4249-8ffc-e63a33c95419","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.747 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_32","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.746355Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.131000Z","msg_id":"87a59b21-669e-4edb-aefb-7eeaa226eac4","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.748 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_33","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.747241Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:16.131000Z","msg_id":"87a59b21-669e-4edb-aefb-7eeaa226eac4","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.748 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_34","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.747574Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.694000Z","msg_id":"5efa94b6-39bb-4b99-9162-4848cbe51868","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.749 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_35","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.748400Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.694000Z","msg_id":"5efa94b6-39bb-4b99-9162-4848cbe51868","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.750 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_36","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.749467Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.695000Z","msg_id":"d928486e-5e27-442e-ac98-47f981a3d461","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.751 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_37","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.750156Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.695000Z","msg_id":"d928486e-5e27-442e-ac98-47f981a3d461","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.751 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_38","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.750491Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.695000Z","msg_id":"779acb1a-3010-4fe4-85de-3351b3fa995b","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.753 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_39","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.751272Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.695000Z","msg_id":"779acb1a-3010-4fe4-85de-3351b3fa995b","msg_type":"comm_close","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.754 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_40","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.751883Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"} 23:17:18.754 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_41","msg_type":"execute_input","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.751997Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"code":"try:\n import ipywidgets as _VSCODE_ipywidgets\n print(\"e976ee50-99ed-4aba-9b6b-9dcd5634d07d:IPyWidgets:\" + _VSCODE_ipywidgets.__version__)\n del _VSCODE_ipywidgets\nexcept:\n pass","execution_count":1},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,101,120,101,99,117,116,101,95,105,110,112,117,116]}],"buffers":[],"channel":"iopub"} 23:17:18.849 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_42","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.847591Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":"25px","justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"50%"},"buffer_paths":[]},"comm_id":"142ec4a7f205410a945ce9165e1bb47f","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,49,52,50,101,99,52,97,55,102,50,48,53,52,49,48,97,57,52,53,99,101,57,49,54,53,101,49,98,98,52,55,102]}],"buffers":[],"channel":"iopub"} 23:17:18.849 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_43","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.848070Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""},"buffer_paths":[]},"comm_id":"e6cff3bc11be4096b2485320e805779c","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,101,54,99,102,102,51,98,99,49,49,98,101,52,48,57,54,98,50,52,56,53,51,50,48,101,56,48,53,55,55,57,99]}],"buffers":[],"channel":"iopub"} 23:17:18.849 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_44","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.848422Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"info","description":"Progress:","description_allow_html":false,"layout":"IPY_MODEL_142ec4a7f205410a945ce9165e1bb47f","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e6cff3bc11be4096b2485320e805779c","tabbable":null,"tooltip":null,"value":0},"buffer_paths":[]},"comm_id":"bdd1fc6f09b04ef5966e4e775288a7db","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,98,100,100,49,102,99,54,102,48,57,98,48,52,101,102,53,57,54,54,101,52,101,55,55,53,50,56,56,97,55,100,98]}],"buffers":[],"channel":"iopub"} 23:17:18.850 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_45","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.849441Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"FloatProgress(value=0.0, bar_style='info', description='Progress:', layout=Layout(height='25px', width='50%'),…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bdd1fc6f09b04ef5966e4e775288a7db"}},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"} 23:17:18.884 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_46","msg_type":"comm_close","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.883644Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{},"comm_id":"bdd1fc6f09b04ef5966e4e775288a7db"},"idents":[{"type":"Buffer","data":[99,111,109,109,45,98,100,100,49,102,99,54,102,48,57,98,48,52,101,102,53,57,54,54,101,52,101,55,55,53,50,56,56,97,55,100,98]}],"buffers":[],"channel":"iopub"} ```

23:17:18.887 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_48","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.886686Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:18.888 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_47","msg_type":"execute_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.885898Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.743000Z","msg_id":"916ca4e6-3b89-4e91-8f5f-6c7a9b9d7b38","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"started":"2024-08-31T15:17:18.751968Z","dependencies_met":true,"engine":"44ca996b-4012-4d50-8721-633ecb536a60","status":"ok"},"content":{"status":"ok","execution_count":0,"user_expressions":{},"payload":[]},"idents":[],"buffers":[],"channel":"shell"}
23:17:18.888 [trace] Executing silently Code (completed) = try:\nimport ipywidgets as _VSCODE_ipywidgets\nprint("e976ee50-99ed-4aba-9b6b-9dcd5634d07d:IPy with 1 output(s)
23:17:18.888 [trace] Determined IPyWidgets Version as undefined
23:17:18.888 [trace] Executing silently Code (idle) = def __VSCODE_inject_module():\ndef __VSCODE_call_function(function, callback, data=None):
23:17:18.888 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"silent":false,"store_history":false,"user_expressions":{},"allow_stdin":true,"stop_on_error":false,"code":"\ndef __VSCODE_inject_module():\n\n    def __VSCODE_call_function(function, callback, data=None):\n        __VSCODE_send_chat_message__(function, data, callback=callback)\n\n    def __VSCODE_send_chat_message__(function, data, callback):\n        requests = {}\n        try:\n            requests = __VSCODE_send_chat_message__.__requests\n        except Exception:\n            __VSCODE_send_chat_message__.__requests = requests\n\n        import uuid as __VSCODE_send_chat_message__uuid\n        import IPython.display as __VSCODE_send_chat_message__ipython_display\n\n        id = str(__VSCODE_send_chat_message__uuid.uuid4())\n        requests[id] = callback\n        data_is_none = data is None\n        __VSCODE_send_chat_message__ipython_display.display({\"application/vnd.vscode.chat_message\": data}, metadata={\"id\":id, \"function\": function, \"dataIsNone\": data_is_none}, raw=True)\n\n        del __VSCODE_send_chat_message__ipython_display\n        del __VSCODE_send_chat_message__uuid\n\n    def __VSCODE_on_chat_message(id, data):\n        requests = {}\n        try:\n            requests = __VSCODE_send_chat_message__.__requests\n        except Exception:\n            __VSCODE_send_chat_message__.__requests = requests\n\n        if id in requests:\n            requests[id](data)\n            del requests[id]\n        else:\n            raise NotImplementedError(f\"Callback not found for message {id}\")\n\n    import sys as __VSCODE_send_chat_message__sys\n    import IPython as __VSCODE_send_chat_message__IPython\n    chat = type(__VSCODE_send_chat_message__IPython)(\"chat\")\n    chat.send_message = __VSCODE_send_chat_message__\n    chat.call_function = __VSCODE_call_function\n    chat.__on_message = __VSCODE_on_chat_message\n    __VSCODE_send_chat_message__sys.modules[\"vscode\"] = type(__VSCODE_send_chat_message__IPython)(\"vscode\")\n    __VSCODE_send_chat_message__sys.modules[\"vscode\"].chat = chat\n    del __VSCODE_send_chat_message__sys\n    del __VSCODE_send_chat_message__IPython\n\n\n__VSCODE_inject_module()\ndel __VSCODE_inject_module\n\n__vsc_ipynb_file__ = \"/Users/~/Desktop/livy-emr.ipynb\"\n%config Completer.use_jedi = False"},"header":{"date":"2024-08-31T15:17:18.888Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}}
23:17:18.891 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_49","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.889768Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:18.892 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_50","msg_type":"execute_input","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:18.889915Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"code":"\ndef __VSCODE_inject_module():\n\n    def __VSCODE_call_function(function, callback, data=None):\n        __VSCODE_send_chat_message__(function, data, callback=callback)\n\n    def __VSCODE_send_chat_message__(function, data, callback):\n        requests = {}\n        try:\n            requests = __VSCODE_send_chat_message__.__requests\n        except Exception:\n            __VSCODE_send_chat_message__.__requests = requests\n\n        import uuid as __VSCODE_send_chat_message__uuid\n        import IPython.display as __VSCODE_send_chat_message__ipython_display\n\n        id = str(__VSCODE_send_chat_message__uuid.uuid4())\n        requests[id] = callback\n        data_is_none = data is None\n        __VSCODE_send_chat_message__ipython_display.display({\"application/vnd.vscode.chat_message\": data}, metadata={\"id\":id, \"function\": function, \"dataIsNone\": data_is_none}, raw=True)\n\n        del __VSCODE_send_chat_message__ipython_display\n        del __VSCODE_send_chat_message__uuid\n\n    def __VSCODE_on_chat_message(id, data):\n        requests = {}\n        try:\n            requests = __VSCODE_send_chat_message__.__requests\n        except Exception:\n            __VSCODE_send_chat_message__.__requests = requests\n\n        if id in requests:\n            requests[id](data)\n            del requests[id]\n        else:\n            raise NotImplementedError(f\"Callback not found for message {id}\")\n\n    import sys as __VSCODE_send_chat_message__sys\n    import IPython as __VSCODE_send_chat_message__IPython\n    chat = type(__VSCODE_send_chat_message__IPython)(\"chat\")\n    chat.send_message = __VSCODE_send_chat_message__\n    chat.call_function = __VSCODE_call_function\n    chat.__on_message = __VSCODE_on_chat_message\n    __VSCODE_send_chat_message__sys.modules[\"vscode\"] = type(__VSCODE_send_chat_message__IPython)(\"vscode\")\n    __VSCODE_send_chat_message__sys.modules[\"vscode\"].chat = chat\n    del __VSCODE_send_chat_message__sys\n    del __VSCODE_send_chat_message__IPython\n\n\n__VSCODE_inject_module()\ndel __VSCODE_inject_module\n\n__vsc_ipynb_file__ = \"/Users/~/Desktop/livy-emr.ipynb\"\n%config Completer.use_jedi = False","execution_count":1},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,101,120,101,99,117,116,101,95,105,110,112,117,116]}],"buffers":[],"channel":"iopub"}
23:17:19.004 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_51","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.003200Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":"25px","justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"50%"},"buffer_paths":[]},"comm_id":"b1e830538dae466da9f2a4056a766b49","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,98,49,101,56,51,48,53,51,56,100,97,101,52,54,54,100,97,57,102,50,97,52,48,53,54,97,55,54,54,98,52,57]}],"buffers":[],"channel":"iopub"}
23:17:19.005 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_52","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.003605Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""},"buffer_paths":[]},"comm_id":"76f8b4fdd4b34f1880641bf9d187792c","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,55,54,102,56,98,52,102,100,100,52,98,51,52,102,49,56,56,48,54,52,49,98,102,57,100,49,56,55,55,57,50,99]}],"buffers":[],"channel":"iopub"}
23:17:19.005 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_53","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.003700Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"info","description":"Progress:","description_allow_html":false,"layout":"IPY_MODEL_b1e830538dae466da9f2a4056a766b49","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_76f8b4fdd4b34f1880641bf9d187792c","tabbable":null,"tooltip":null,"value":0},"buffer_paths":[]},"comm_id":"4834265a59434b0db5fde6489f42c7e5","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,52,56,51,52,50,54,53,97,53,57,52,51,52,98,48,100,98,53,102,100,101,54,52,56,57,102,52,50,99,55,101,53]}],"buffers":[],"channel":"iopub"}
23:17:19.005 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_54","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.004663Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"FloatProgress(value=0.0, bar_style='info', description='Progress:', layout=Layout(height='25px', width='50%'),…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4834265a59434b0db5fde6489f42c7e5"}},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"}
23:17:19.848 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_55","msg_type":"comm_close","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.847393Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{},"comm_id":"4834265a59434b0db5fde6489f42c7e5"},"idents":[{"type":"Buffer","data":[99,111,109,109,45,52,56,51,52,50,54,53,97,53,57,52,51,52,98,48,100,98,53,102,100,101,54,52,56,57,102,52,50,99,55,101,53]}],"buffers":[],"channel":"iopub"}
23:17:19.849 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_56","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.848163Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","date":"2024-08-31T15:17:18.888000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stderr","text":"An error was encountered:\nunknown magic command 'config'\nUnknownMagic: unknown magic command 'config'\n\n"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,101,114,114]}],"buffers":[],"channel":"iopub"}
23:17:19.851 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_58","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.850454Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.851 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_57","msg_type":"execute_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.849493Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:18.888000Z","msg_id":"d6b43a61-3d07-4cf7-9d7d-df58bc052535","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"started":"2024-08-31T15:17:18.889903Z","dependencies_met":true,"engine":"44ca996b-4012-4d50-8721-633ecb536a60","status":"ok"},"content":{"status":"ok","execution_count":0,"user_expressions":{},"payload":[]},"idents":[],"buffers":[],"channel":"shell"}
23:17:19.851 [trace] Executing silently Code (completed) = def __VSCODE_inject_module():\ndef __VSCODE_call_function(function, callback, data=None): with 2 output(s)
23:17:19.852 [debug] Requesting Kernel info
23:17:19.852 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{},"header":{"date":"2024-08-31T15:17:19.852Z","msg_id":"d786b537-9492-4d4c-9ae7-6315bd6c9b22","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}}
23:17:19.852 [trace] Got Kernel info
23:17:19.852 [trace] End running kernel initialization, now waiting for idle
23:17:19.852 [trace] Waiting for idle on (kernel): 541de6cd-5a2e-40df-9ffe-912511b766f1 -> idle
23:17:19.853 [trace] Finished waiting for idle on (kernel): 541de6cd-5a2e-40df-9ffe-912511b766f1 -> idle
23:17:19.853 [trace] End running kernel initialization, session is idle
23:17:19.854 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"code":"__file__.","cursor_pos":9},"header":{"date":"2024-08-31T15:17:19.853Z","msg_id":"2a06149e-d6ed-4f7b-9680-0803d217f6b0","msg_type":"complete_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"parent_header":{}}
23:17:19.854 [trace] Registering Kernel Completion Provider from kernel PySpark (Python 3.8.19) for language python
23:17:19.854 [trace] IPyWidgetScriptSource.initialize
23:17:19.855 [info] Kernel successfully started
23:17:19.855 [warn] No nbextensions folder found for kernel .undefined.sparkmagic.kernels.pysparkkernel.pysparkkernel./Users/~/miniforge3/envs/sparkmagictest/python./Users/~/miniforge3/envs/sparkmagictest/python.-m#sparkmagic.kernels.pysparkkernel.pysparkkernel
23:17:19.857 [trace] Cell Index:0 sent to kernel
23:17:19.858 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"silent":false,"store_history":true,"user_expressions":{},"allow_stdin":true,"stop_on_error":false,"code":"%%configure -f\n{\n    \"driverCores\": 1,\n    \"driverMemory\": \"3g\",\n    \"executorCores\": 1,\n    \"executorMemory\": \"3g\",\n    \"numExecutors\": 2,\n    \"conf\" : {\n        \"spark.dynamicAllocation.enabled\":  \"true\",\n        \"spark.dynamicAllocation.initialExecutors\":  2,\n        \"spark.dynamicAllocation.minExecutors\": 2,\n        \"spark.dynamicAllocation.maxExecutors\":  4,\n        \"spark.sql.files.ignoreMissingFiles\": \"true\"\n    }\n}"},"header":{"date":"2024-08-31T15:17:19.858Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"cellId":"vscode-notebook-cell:/Users/~/Desktop/livy-emr.ipynb#W0sZmlsZQ%3D%3D"},"parent_header":{}}
23:17:19.859 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_60","msg_type":"kernel_info_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.854467Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.852000Z","msg_id":"d786b537-9492-4d4c-9ae7-6315bd6c9b22","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"status":"ok","protocol_version":"5.3","implementation":"PySpark","implementation_version":"1.0","language_info":{"name":"pyspark","mimetype":"text/x-python","codemirror_mode":{"name":"python","version":3},"file_extension":".py","pygments_lexer":"python3"},"banner":"Python 3.8.19 | packaged by conda-forge | (default, Mar 20 2024, 12:49:57) \nType 'copyright', 'credits' or 'license' for more information\nIPython 8.12.3 -- An enhanced Interactive Python. Type '?' for help.\n","help_links":[{"text":"Python Reference","url":"https://docs.python.org/3.8"},{"text":"IPython Reference","url":"https://ipython.org/documentation.html"},{"text":"NumPy Reference","url":"https://docs.scipy.org/doc/numpy/reference/"},{"text":"SciPy Reference","url":"https://docs.scipy.org/doc/scipy/reference/"},{"text":"Matplotlib Reference","url":"https://matplotlib.org/contents.html"},{"text":"SymPy Reference","url":"http://docs.sympy.org/latest/index.html"},{"text":"pandas Reference","url":"https://pandas.pydata.org/pandas-docs/stable/"}]},"idents":[],"buffers":[],"channel":"shell"}
23:17:19.861 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_59","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.853737Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.852000Z","msg_id":"d786b537-9492-4d4c-9ae7-6315bd6c9b22","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.861 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_61","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.855538Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.852000Z","msg_id":"d786b537-9492-4d4c-9ae7-6315bd6c9b22","msg_type":"kernel_info_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.863 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_62","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.857034Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.853000Z","msg_id":"2a06149e-d6ed-4f7b-9680-0803d217f6b0","msg_type":"complete_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.973 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_64","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.972415Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.853000Z","msg_id":"2a06149e-d6ed-4f7b-9680-0803d217f6b0","msg_type":"complete_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.973 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_63","msg_type":"complete_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.971988Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.853000Z","msg_id":"2a06149e-d6ed-4f7b-9680-0803d217f6b0","msg_type":"complete_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"matches":[],"cursor_end":9,"cursor_start":9,"metadata":{"_jupyter_types_experimental":[]},"status":"ok"},"idents":[],"buffers":[],"channel":"shell"}
23:17:19.973 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_65","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.973323Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"busy"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:17:19.974 [trace] Start cell 0 execution @ 1725117439974 (clear output)
23:17:19.974 [debug] Kernel acknowledged execution of cell 0 @ 1725117439974
23:17:19.983 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_66","msg_type":"execute_input","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:19.973406Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"code":"%%configure -f\n{\n    \"driverCores\": 1,\n    \"driverMemory\": \"3g\",\n    \"executorCores\": 1,\n    \"executorMemory\": \"3g\",\n    \"numExecutors\": 2,\n    \"conf\" : {\n        \"spark.dynamicAllocation.enabled\":  \"true\",\n        \"spark.dynamicAllocation.initialExecutors\":  2,\n        \"spark.dynamicAllocation.minExecutors\": 2,\n        \"spark.dynamicAllocation.maxExecutors\":  4,\n        \"spark.sql.files.ignoreMissingFiles\": \"true\"\n    }\n}","execution_count":1},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,101,120,101,99,117,116,101,95,105,110,112,117,116]}],"buffers":[],"channel":"iopub"}
23:17:21.441 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_67","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:17:21.439013Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","date":"2024-08-31T15:17:19.858000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stdout","text":"Starting Spark application\n"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,111,117,116]}],"buffers":[],"channel":"iopub"}
23:18:23.627 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_68","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:23.623393Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<table>\n<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>User</th><th>Current session?</th></tr><tr><td>1</td><td>spark-c1dd087d3a64446d82cbf00819a008b4</td><td>pyspark</td><td>idle</td><td><a target=\"_blank\" href=\"https://pre-emr-spark-ui-cn-hangzhou.data.aliyun.com/runtime/jobs/?token=VHpIT09lUkZJZk9aTVJpSCtwNENPaVVDUGlSY0RDU21CVGlTcGpMSC9OMGUyN01ZaUtlbS9aWlBBazFsd3k3alZhQU51NzBDcGdyeDBlSWhZbnI0N2hqUGN0NzViem5FZUFIV1pNeVgvRWEwWUdzeG1RVFpEWXpobEk2TlNUMUE5ay9EbEZvOGRuNVc3TUt6K1ZWNDJSeEc1U0dnRWZCM2dnMm5DSzdVNXI0a2FDakhlRW5CdkFtaUNtTWlTOGwrWTBGenlqb0FSWUlpS3JZR2c2REt5d0Zua210ZS9raERWWHhqa1RyNStxNkNobkxPekZjQ1p4VVBHck1iYXBMdis1VGx2anVjMzJkemZOVE96OFlHYkdMeGd5VFcxNzAyOEdQVVBSODdtd205c1hTYmovelovZExoNUZtczhpQ1RrY2xZTFltemtFcUNabi96d3BSd3dONW5yVVlFZmp4d0pKSktnU0hwTDg2M2xjdmFwenJTNXFsZUhSdWFFVm5ac2tISGd2RzVFNXM9\">Link</a></td><td></td><td>None</td><td>✔</td></tr></table>"},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"}
23:18:23.735 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_69","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:23.733292Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"2.0.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border_bottom":null,"border_left":null,"border_right":null,"border_top":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":"25px","justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"50%"},"buffer_paths":[]},"comm_id":"2fcf15deb0ad4ad0b2ae0735fff7f54b","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,50,102,99,102,49,53,100,101,98,48,97,100,52,97,100,48,98,50,97,101,48,55,51,53,102,102,102,55,102,53,52,98]}],"buffers":[],"channel":"iopub"}
23:18:23.736 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_70","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:23.734030Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"2.0.0","_view_name":"StyleView","bar_color":null,"description_width":""},"buffer_paths":[]},"comm_id":"d10a2d529c894fa7a5138b71123b1dcd","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,100,49,48,97,50,100,53,50,57,99,56,57,52,102,97,55,97,53,49,51,56,98,55,49,49,50,51,98,49,100,99,100]}],"buffers":[],"channel":"iopub"}
23:18:23.736 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_71","msg_type":"comm_open","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:23.734149Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"version":"2.1.0"},"content":{"data":{"state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"2.0.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"2.0.0","_view_name":"ProgressView","bar_style":"info","description":"Progress:","description_allow_html":false,"layout":"IPY_MODEL_2fcf15deb0ad4ad0b2ae0735fff7f54b","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d10a2d529c894fa7a5138b71123b1dcd","tabbable":null,"tooltip":null,"value":0},"buffer_paths":[]},"comm_id":"943190c492df470a989b7c1989864af5","target_name":"jupyter.widget","target_module":null},"idents":[{"type":"Buffer","data":[99,111,109,109,45,57,52,51,49,57,48,99,52,57,50,100,102,52,55,48,97,57,56,57,98,55,99,49,57,56,57,56,54,52,97,102,53]}],"buffers":[],"channel":"iopub"}
23:18:23.737 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_72","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:23.735179Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"FloatProgress(value=0.0, bar_style='info', description='Progress:', layout=Layout(height='25px', width='50%'),…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"943190c492df470a989b7c1989864af5"}},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"}
23:18:26.348 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_73","msg_type":"comm_close","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.344916Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{},"comm_id":"943190c492df470a989b7c1989864af5"},"idents":[{"type":"Buffer","data":[99,111,109,109,45,57,52,51,49,57,48,99,52,57,50,100,102,52,55,48,97,57,56,57,98,55,99,49,57,56,57,56,54,52,97,102,53]}],"buffers":[],"channel":"iopub"}
23:18:26.352 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_74","msg_type":"stream","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.347812Z","version":"5.3"},"parent_header":{"msg_type":"execute_request","date":"2024-08-31T15:17:19.858000Z","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","version":"5.2","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968"},"metadata":{},"content":{"name":"stdout","text":"SparkSession available as 'spark'.\n"},"idents":[{"type":"Buffer","data":[115,116,114,101,97,109,46,115,116,100,111,117,116]}],"buffers":[],"channel":"iopub"}
23:18:26.353 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_75","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.352135Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"Current session configs: <tt>{'conf': {'spark.dynamicAllocation.enabled': 'true', 'spark.dynamicAllocation.initialExecutors': 2, 'spark.dynamicAllocation.minExecutors': 2, 'spark.dynamicAllocation.maxExecutors': 4, 'spark.sql.files.ignoreMissingFiles': 'true'}, 'driverCores': 1, 'driverMemory': '3g', 'executorCores': 1, 'executorMemory': '3g', 'numExecutors': 2, 'kind': 'pyspark'}</tt><br>"},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"}
23:18:26.705 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_76","msg_type":"display_data","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.703776Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"data":{"text/plain":"<IPython.core.display.HTML object>","text/html":"<table>\n<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>User</th><th>Current session?</th></tr><tr><td>1</td><td>spark-c1dd087d3a64446d82cbf00819a008b4</td><td>pyspark</td><td>idle</td><td><a target=\"_blank\" href=\"https://pre-emr-spark-ui-cn-hangzhou.data.aliyun.com/runtime/jobs/?token=NnpubGVsc3ppQWdubjNBWWdYdjVLT0xhNm81bmV6b2Z1dDZJdVBpSTNUdUNsNWJOUEl0VFQ5ejZoSy9yRDgybU9remJGQXhlMXU5NGRkT1NDdTNWNE1JNXBwYzIrVTdZQ0JJY0Z1TFdsTmpWMS9vM21nK2tldDVJTzRNbm0yMWQ0ZS9hUWJtbXh5eHFLSnFVUzF1M2hYRW1UVzRjQ3pJOCtYTlhySWxrMEtLckg3N28yQkxKbzlyQXVicEVyb0lEeUM2T3BldytyUHVOemRVZDUwV1luanF2Y1FpQTFFRE00NUp4SGRqWSs2a3BsUVVkaExPVHBjSkFocUxLZGpXYWFvRm5OanlhSDhlRXUzbFVGTGI1M0hOc1dQczhjTEUySk9mL3lJMmVaWWlSUHl2eGlvUkFUd2xzZkxuQWordkx3eW9ROVN2akg0dE1OR2RvRG1sbDRDMnB3dDFaUWtNRVRsck1Icm5nOGVCbjM4aGpZWDdhUjAwVHM0azBleGR0NG5TWmtYekl5Y3M9\">Link</a></td><td></td><td>None</td><td>✔</td></tr></table>"},"metadata":{},"transient":{}},"idents":[{"type":"Buffer","data":[100,105,115,112,108,97,121,95,100,97,116,97]}],"buffers":[],"channel":"iopub"}
23:18:26.708 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_78","msg_type":"status","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.707716Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{},"content":{"execution_state":"idle"},"idents":[{"type":"Buffer","data":[107,101,114,110,101,108,46,52,52,99,97,57,57,54,98,45,52,48,49,50,45,52,100,53,48,45,56,55,50,49,45,54,51,51,101,99,98,53,51,54,97,54,48,46,115,116,97,116,117,115]}],"buffers":[],"channel":"iopub"}
23:18:26.710 [trace] [AnyMessage Event] [recv] [58066] {"header":{"msg_id":"fbace886-d2a65ca6647108231dd95353_58066_77","msg_type":"execute_reply","username":"~","session":"fbace886-d2a65ca6647108231dd95353","date":"2024-08-31T15:18:26.706799Z","version":"5.3"},"parent_header":{"date":"2024-08-31T15:17:19.858000Z","msg_id":"a80e0396-9a0f-4669-8c85-a64e1c26c4b5","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"started":"2024-08-31T15:17:19.973395Z","dependencies_met":true,"engine":"44ca996b-4012-4d50-8721-633ecb536a60","status":"ok"},"content":{"status":"ok","execution_count":1,"user_expressions":{},"payload":[]},"idents":[],"buffers":[],"channel":"shell"}
23:18:26.711 [trace] Cell 0 completed in 66.736s (start: 1725117439974, end: 1725117506710)
23:18:26.711 [trace] Cell 0 executed successfully
23:20:16.108 [debug] Handle Execution of Cells 1 for ~/Desktop/livy-emr.ipynb
23:20:16.109 [trace] Cell Index:1 sent to kernel
23:20:16.110 [trace] [AnyMessage Event] [send] [58066] {"buffers":[],"channel":"shell","content":{"silent":false,"store_history":true,"user_expressions":{},"allow_stdin":true,"stop_on_error":false,"code":"spark"},"header":{"date":"2024-08-31T15:20:16.110Z","msg_id":"54995aa1-abf5-449f-8447-57489669fb43","msg_type":"execute_request","session":"7e4ba6c2-8f38-4248-b77a-fc7954144968","username":"8a029f6d-8d16-4f86-ba6c-1d63fe1cb654","version":"5.2"},"metadata":{"cellId":"vscode-notebook-cell:/Users/~/Desktop/livy-emr.ipynb#W1sZmlsZQ%3D%3D"},"parent_header":{}}

microsoft / vscode-jupyter