Open tor-axbit opened 7 months ago
Hi @tor-axbit, Sorry for the late reply. Can you confirm if this issue is sporadic or reproducible, and if it is still occurring? It seems that the error you encountered occurs during the initiation of the Spark session, particularly after an unexpected closure of VS Code, such as closing VS Code while a cell is running. We are currently unable to reproduce this error on our end.
Normally, closing the VS Code window should trigger a restart of the session. If the session does not restart properly, it may lead to AttributeError: 'NoneType' object has no attribute 'sql' when you attempt to run a Spark SQL query, as the Spark context might not be properly initialized. It would be helpful to ensure that the Spark session is fully restarted after reopening VS Code. Let us know if the problem still exist.
I have two Spark notebooks running in VS Code, latest extension version, Spark runtime 1-2. The notebooks mostly run fine, but on occasion, for example if accidentally closing VS Code while runtime could be running a cell, I get
AttributeError: 'NoneType' object has no attribute 'sql'
when running a cell with Spark SQL query after reopening.Redacted copypaste of Spark lighter log