search

microsoft / SynapseVSCode

this is the repo of the Synapse VS Code extension for Microsoft Fabric
MIT License
23 stars 4 forks source link

Arbitrary notebook runtime problem #28

Open tor-axbit opened 7 months ago

tor-axbit commented 7 months ago

I have two Spark notebooks running in VS Code, latest extension version, Spark runtime 1-2. The notebooks mostly run fine, but on occasion, for example if accidentally closing VS Code while runtime could be running a cell, I get AttributeError: 'NoneType' object has no attribute 'sql' when running a cell with Spark SQL query after reopening.

Redacted copypaste of Spark lighter log

08:39:05,769 root INFO current_directory c:\temp\local_work_2\d451e020-dc9b-4ff4-a34e-fe1daca7e420\SynapseNotebook\285d45ce-4634-47e9-ac3f-faf0e197fa41\<NOTEBOOK NAME>
08:39:05,769 root INFO workspace_path c:\temp\local_work_2\d451e020-dc9b-4ff4-a34e-fe1daca7e420
08:39:05,769 root INFO log_path c:\temp\local_work_2\d451e020-dc9b-4ff4-a34e-fe1daca7e420\logs\285d45ce-4634-47e9-ac3f-faf0e197fa41
08:39:05,769 root INFO Using synapse remote kernel ...
08:39:05,769 root INFO Should attach session in dev mode False
08:39:05,852 root INFO Starting session edfed829-3f0d-4f5d-8c2d-951a5e5cffd8...
08:39:08,553 root INFO Getting refresh token...
08:39:10,73 root INFO https://pbipneu7-northeurope.pbidedicated.windows.net/webapi/capacities/AB7115F4-18B2-4746-A830-A08C71D0CC02/workloads/Notebook/Data/Direct/sparklighter/api/tjs/versions/2022-04-30/artifacts/285d45ce-4634-47e9-ac3f-faf0e197fa41/sessions/edfed829-3f0d-4f5d-8c2d-951a5e5cffd8
08:39:11,414 root INFO <session_management.SessionStatus object at 0x000001ED9C089AB0>
08:39:11,414 root ERROR Failed to initialize Spark Lighter variables. Create session failed {'code': 'InternalError', 'subCode': 0, 'message': 'An internal error occurred.', 'timeStamp': '2024-02-06T07:39:11.3024525Z', 'httpStatusCode': 500, 'hresult': -2147467259, 'details': [{'code': 'RootActivityId', 'message': 'da14e9c2-96b5-4dd1-8758-cc136a38eb3f'}, {'code': 'Param1', 'message': 'Response status code does not indicate success: 430 ().'}]}
08:39:11,414 root INFO Registering Spark Lighter magics for IPython...
08:39:11,415 root INFO Registered Spark Lighter magics for IPython.
PotatoLu666 commented 4 months ago

Hi @tor-axbit, Sorry for the late reply. Can you confirm if this issue is sporadic or reproducible, and if it is still occurring? It seems that the error you encountered occurs during the initiation of the Spark session, particularly after an unexpected closure of VS Code, such as closing VS Code while a cell is running. We are currently unable to reproduce this error on our end.

Normally, closing the VS Code window should trigger a restart of the session. If the session does not restart properly, it may lead to AttributeError: 'NoneType' object has no attribute 'sql' when you attempt to run a Spark SQL query, as the Spark context might not be properly initialized. It would be helpful to ensure that the Spark session is fully restarted after reopening VS Code. Let us know if the problem still exist.