Open timsu92 opened 2 months ago
timsu92
commented
2 months ago This is another log which I ran some cells, scroll them away, and then I see requests on cells (what's this?) and the kernel were both disposed. Althought the log said I deleted some cells, I didn't. I was just scrolling throught the notebook.
XC-Li
commented
2 months ago Hi, @DonJayamanne I got the same error these days. It seems to happen on new created jupyter notebooks. And only happens when I run jupyter notebook on remote ubuntu server, local run is fine. @timsu92 , please let me know if you found any solution, thanks.
Version info: Python version: 3.12.3 Remote server version: Ubuntu 22.04.4 LTS Local system: MacOS 14.5 (23F79)
VS Code Version: Version: 1.89.1 (Universal) Commit: dc96b837cf6bb4af9cd736aa3af08cf8279f7685 Date: 2024-05-07T05:14:24.611Z (3 wks ago) Electron: 28.2.8 ElectronBuildId: 27744544 Chromium: 120.0.6099.291 Node.js: 18.18.2 V8: 12.0.267.19-electron.0 OS: Darwin arm64 23.5.0
All VS Code extension are most recent version.
jupyter package version: IPython : 8.24.0 ipykernel : 6.29.3 ipywidgets : not installed jupyter_client : 8.6.2 jupyter_core : 5.7.2 jupyter_server : not installed jupyterlab : not installed nbclient : not installed nbconvert : not installed nbformat : not installed notebook : not installed qtconsole : not installed traitlets : 5.14.3
My sample code, which will crash in about 3 minutes:
for i in range(6000): time.sleep(1) print(i)
Here is the log:
DonJayamanne
commented
2 months ago 11:33:52.657 [debug] Disposing kernel associated with ~/analysis/su_notebooks/temp/loop_only.ipynb, isClosed=true
Based on the logs, either a cell that was being executed was deleted or the notebook itself was closed. Thus, this is by design
XC-Li
commented
2 months ago Hi Don, Thanks for replying. I didn't close the notebook or delete the cell while it's running, why the notebook would think the cell is deleted or notebook is closed?
I tried to run the notebook again, and it think the cell was deleted (which wasn't) this time:
Here is full log from starting the VS code:
XC-Li
commented
2 months ago 11:33:52.657 [debug] Disposing kernel associated with ~/analysis/su_notebooks/temp/loop_only.ipynb, isClosed=true
Based on the logs, either a cell that was being executed was deleted or the notebook itself was closed. Thus, this is by design
Hi @DonJayamanne , I have posted another full log on this error. Please take a look.
timsu92
commented
2 months ago @timsu92 , please let me know if you found any solution, thanks.
@XC-Li Not yet found any thought🥲 Since my problem sometimes looks like that in the new issue, and sometimes not, I'm writing on both issues depending on what I see.
DonJayamanne
commented
2 months ago @timsu92 @XC-Li Looks like both issues are related and the same
23:29:53.226 [info] Cell 21 completed in 0.003s (start: 1716910193223, end: 1716910193226)
23:30:43.408 [info] Execution of code ms-toolsai.jupyter-1 completed in 46ms
23:31:25.815 [info] Disposing request as the cell (7) was deleted ~/quantum_computing/final/final.ipynb
23:31:25.816 [info] Disposing request as the cell (9) was deleted ~/quantum_computing/final/final.ipynbFor some reason some cells are being flagged as bing deleted.
DonJayamanne
commented
2 months ago @XC-LI @timsu92
print("Hello World")Please let me know the results for each of the above. Thanks
timsu92
commented
2 months ago @timsu92 @XC-Li Looks like both issues are related and the same
23:29:53.226 [info] Cell 21 completed in 0.003s (start: 1716910193223, end: 1716910193226) 23:30:43.408 [info] Execution of code ms-toolsai.jupyter-1 completed in 46ms 23:31:25.815 [info] Disposing request as the cell (7) was deleted ~/quantum_computing/final/final.ipynb 23:31:25.816 [info] Disposing request as the cell (9) was deleted ~/quantum_computing/final/final.ipynbFor some reason some cells are being flagged as bing deleted.
Yes, in this second log I posted, the cell looks like being deleted. However, in the first log, only the kernel was disposed. I'm trying to see what makes these two different outcomes.
XC-Li
commented
2 months ago @XC-Li @timsu92
- do you run into this issue when running another notebook?
- Can you try running a different notebook with each cell containing
print("Hello World")- Please can you disable ALL extensions except for Jupyter and Python (i.e. disable Pylance, and all other extensions)
Please let me know the results for each of the above. Thanks
Hi @DonJayamanne , thanks for your help. I may found the cause of this issue. It's caused by this plugin called 'Continue': https://marketplace.visualstudio.com/items?itemName=Continue.continue
This plugin is disabled and required to reload window to apply update this morning, and I found all the notebooks can run normally. Then I reload the window and found the error is back. Then I disabled this plugin and the error is gone. @timsu92 You can check if you have this plugin too, or try disable all other plugins and see if the error persists.
timsu92
commented
2 months ago Hi, @XC-Li ! I have Continue installed as well, and disabling it truly solved the problem. However, re-enabling it doesn't trigger the problem as well. I guess this problem may be somewhat probabilistic, as I successfully executed the same notebook yesterday. Yesterday, I also tried disabling all the plugins except WSL, language pack, Jupyter plugins and Python, and it was successful as well.
timsu92
commented
2 months ago After testing more, this is what I got: enabling Contiinue doesn't always trigger the error, but disabling it with all other plugins enabled works like a charm.
If I date back Continue to version v0.8.27 (which was released 3 weeks ago), I'm not facing any errors more. @XC-Li Can you confirm?
XC-Li
commented
2 months ago After testing more, this is what I got: enabling Contiinue doesn't always trigger the error, but disabling it with all other plugins enabled works like a charm.
If I date back Continue to version
v0.8.27(which was released 3 weeks ago), I'm not facing any errors more. @XC-Li Can you confirm?
Yes, downgrade Continue to 0.8.27 solves the problem. thanks.
DonJayamanne
commented
2 months ago Thank you for getting back, I would file an issue in the Continue repo . Given this is caused by a 3rd party extension, I'm closing this issue.
timsu92
commented
2 months ago Have we posted an issue there yet?
DonJayamanne
commented
2 months ago No I haven't
timsu92
commented
2 months ago OK no problem! I'm trying to record a video for their issue template, but the error didn't show up today.
edgBR
commented
1 month ago Hi,
I am having the same issue:
[info] Disposing request as the cell (-1) was deleted
I havent deleted any cell, but my notebooks flickers everytime. This doesnt happen in jupyter lab.
My version of VSCode:
Version: 1.90.2 (user setup) Commit: 5437499feb04f7a586f677b155b039bc2b3669eb Date: 2024-06-18T22:34:26.404Z Electron: 29.4.0 ElectronBuildId: 9728852 Chromium: 122.0.6261.156 Node.js: 20.9.0 V8: 12.2.281.27-electron.0 OS: Windows_NT x64 10.0.19045
And Im running VScode with AzureML extension.
timsu92
commented
1 month ago Hi, @edgBR !
In the issue in Continue #1463, they all face the problem when using remotely, including people here.
I managed to use them by downgrading Continue to version v0.8.27. Maybe you can give it a try!
tommilaj
commented
1 month ago I am facing the same issue, 11:42:40.721 [info] Disposing request as the cell (-1) was deleted <.ipynb path>, however, I don't have Continue installed. I've only faced this using Azure ML remote compute instances, can't replicate locally with the same VSCode configuration. I have identified that the kernel is interrupted when the file is saved, either by autosave or manually. Disabling autosave keeps the kernel running, even with changing context within the editor. This I can replicate with a new notebook. Debug logs do not have any other information.
I've encountered this now in two completely different Azure environments, both of which occurred after a while working on the project: some day when you reconnect to the instance, the issue is there. Before the first occurrance, autosave has worked normally. I've tried
without luck. As if I've just accidentally enabled some keybind feature or similar, after which the notebooks just think they are deleting cells on save.
@DonJayamanne can this be reopened, since it is not only related to the mentioned extension.
bjunkerm
commented
1 month ago I can confirm, I have the same problems as @tommilaj also in the settings of Azure ML compute instances. As described, disabling the auto-safe makes the problem disappear.
DonJayamanne
commented
1 month ago @bjunkerm @tommilaj @edgBR Looks like all of you are using azure ML and thats most likely (some how) causing this issue.
Its possible that the same notebook file is opened in Jupyter Lab or the like and making updates to the ipynb file hence causing this issue.
Updates to the ipynb file from other applicatinos can cause issues and unfrotunately theres not much we can do about this, except ask users to use only one tool to manage/edit ipynb files.
Also, please can you disable all extensions except for Jupyter, Python, and try again.
ReverieTides
commented
1 month ago Same scenario for me too, using azure ml.
[info] Disposing request as the cell (-1) was deleted
I've tried disabling all other addons besides Jupyer and Python and tried numerous rollbacks/reinstalls of vscode and various extension combos, with no success.
edit: No other programs / kernels have the notebook open, I've tested with a brand new file too.
DonJayamanne
commented
1 month ago @tbombach @sevillal @shsuman Are you aware of any back end service that might be running in the background that would be updating the ipynb file. Given this is only reported on Azure ML, I suspect something is going on with some other service that could be updating the ipynb file.
shsuman
commented
1 month ago @DonJayamanne Not exactly. Can you please let me know whether this is happening from Azure ML Studio click out or Azure AI Studio click out ? In the meantime, lets sync up with Azure ML team offline to shed some more light on it
DonJayamanne
commented
1 month ago e Not exactly. Can you please let me know whether this is happening from Azure ML Studio click out or Azure AI Studio click out ?
I have no idea, please can you check
shsuman
commented
1 month ago @edgBR @timsu92 @tommilaj @ReverieTides @bjunkerm Can you please let me know whether you are using Compute instances click out (Azure ML Studio) or Environment containers click out (Azure AI Studio) ?
tommilaj
commented
1 month ago @shsuman I am not sure what the up-to-date naming convention is. I'm using Azure Machine Learning workspace resource, and a Compute instance created within that workspace, which I've connected using the VSCode ms-toolsai.vscode-ai-remote extension. The portal studio is titled Azure AI | Machine Learning Studio, being not very informative.
I found out the saving causing the interruption by chance, when I tried disabling editor configurations and enabling them one by one. Using interactive cells on regular .py files saving has no effect.
Let me know if there is anything I can do to help you. Will try responding within a day or two.
bjunkerm
commented
1 month ago @shsuman Same here, I am working on a compute instance in an Azure Machine Learning workspace.
timsu92
commented
1 month ago @edgBR @timsu92 @tommilaj @ReverieTides @bjunkerm Can you please let me know whether you are using Compute instances click out (Azure ML Studio) or Environment containers click out (Azure AI Studio) ?
I'm not using Azure ML, but using VSCode on Windows and connect to local WSL2 (Ubuntu 22.04), and that's why I think I'm running Jupyter remotely.
DonJayamanne
commented
1 month ago https://github.com/microsoft/vscode-jupyter/issues/15716#issuecomment-2225386832
@shsuman Looks like this might not be an AzML issue after all.
edgBR
commented
1 month ago Hi @DonJayamanne
I am currently OOO but I was using AzureML compute instance in Azure Machine Learning Studio. As @tommilaj I'm using the VScode extension to connect to the compute instance. However I also launched VScode in the browser and I had exactly the same problem.
BR E
DonJayamanne
commented
1 month ago However I also launched VScode in the browser and I had exactly the same problem.
Were you connected to Jupyter in the browser? I.e. were you connected to a person remote jupyter server, or azml, or was it code spaces?
edgBR
commented
1 month ago Hi @DonJayamanne
Again, just AzureML, basically in the UI of compute instances you have the option to launch jupyterlab, VScode and VSCode in the browser (https://learn.microsoft.com/en-us/azure/machine-learning/how-to-launch-vs-code-remote?view=azureml-api-2&tabs=vscode-web). When using VSCode option you need the azureML extension in VSCode, when you use the browser VSCode you don't need that extension. In any of the cases I have the same problem.
shsuman
commented
1 month ago en using VSCode option you need the azureML extension in VSCode, when you use the browser VSCode you don't need that extension. In any of the cases I have the same problem.
@DonJayamanne seems like a generic remote workspace problem. I suggest looking if you can repro in WSL
DonJayamanne
commented
1 month ago @edgBR @tommilaj @ReverieTides @bjunkerm I'm sorry you continue to run into this issue and thanks for your patience.
I also noticed that you hadn't provided logs, perhaps your scenarios could be different from the case where logs were provided (as that was due to the cursor extension).
Please can you share the logs when you run into this, along with screenshots of any error messages dispalyed Please could you enable logging as follows:
Jupyter->LoggingverboseJupyter output panel. (use the command Jupyter: Show Output to get there)Can you try the following
print statements and possibly time.sleepFYI, I haven't been able to repro this at my end neither with WSL nor with Azure ML compute.
ReverieTides
commented
1 month ago @edgBR @tommilaj @ReverieTides @bjunkerm I'm sorry you continue to run into this issue and thanks for your patience.
I also noticed that you hadn't provided logs, perhaps your scenarios could be different from the case where logs were provided (as that was due to the cursor extension).
Please can you share the logs when you run into this, along with screenshots of any error messages dispalyed Please could you enable logging as follows:
- Open settings & go into
Jupyter->Logging- Change the value to
verbose- Reload VS Code,
- Attempt to repro the issue & then copy the output from the
Jupyteroutput panel. (use the commandJupyter: Show Outputto get there)Can you try the following
- Disable auto save & try to repro again
- Does this repro with a simple notebook that only contains
time.sleepWhen exactly do you run into this issue
- Do you run into this only when hitting save or leaving a notebook running for a while and then coming back?
FYI, I haven't been able to repro this at my end neither with WSL nor with Azure ML compute.
Perhaps you can share a very simple notebook as well (that repro this issue)
- Please try to ensure this is a minimal notebook, without using too many packages and the like (as that will only delay the setup and complicate it)
- Provide the exact steps (is it just running all notebooks)
Sorry this is a mess, I'm not experienced with contributing debugging issues.
Here are my logs:
This was the code used to generate the logs.
`import time
for i in range(1000): print(i) time.sleep(1)`
Disabling auto-save stops the issue from happening automatically, but the issue still occurs whenever the file is saved manually when a cell is running.
ReverieTides
commented
1 month ago https://github.com/user-attachments/assets/52afeb4b-23b5-4545-9ca9-ac7b641c8622
Here is a small screen grab of the test
melissa-barca
commented
3 weeks ago I think I've been seeing this same bug. I ran the extension bisect and it identified Jupyter Cell Tags (0.1.9) as the issue. Disabling it or downgrading it to 0.1.8 seems to resolve the issue.
edgBR
commented
3 weeks ago I think I've been seeing this same bug. I ran the extension bisect and it identified Jupyter Cell Tags (0.1.9) as the issue. Disabling it or downgrading it to 0.1.8 seems to resolve the issue.
Hi @melissa-barca I tried to downgrade or disable but I am still having the same problem.
BR E
DonJayamanne
commented
3 weeks ago I know this is not a solution, but merely a very bad work around.
Please disable auto save feature and try to avoid saving whilst executing cells in a notebook
DonJayamanne
commented
2 weeks ago Looks like saving of notebooks somehow causes the models to go out of sync.
amunger
commented
1 week ago Unfortunately, none of us have been able to repro this issue, but we've added in some additional logging in vscode insiders to narrow it down.
If you can still repro this could you:
Developer: set log level and select tracewindow logs from the output panel, as well as the notebook and jupyter logs, and post them all here
Applies To
What happened?
output > JupyterNO notebook document for 'file://*.ipynb'. Meanwhile, the restart kernel button turns gray, and the Go To button at the top keeps on spinning.I found an old issue which was resolved by reverting to an older version of Pylance. However it seems not working this time.
VS Code Version
版本: 1.89.1 (system setup) 認可: dc96b837cf6bb4af9cd736aa3af08cf8279f7685 日期: 2024-05-07T05:13:33.891Z Electron: 28.2.8 ElectronBuildId: 27744544 Chromium: 120.0.6099.291 Node.js: 18.18.2 V8: 12.0.267.19-electron.0 OS: Windows_NT x64 10.0.22631
Jupyter Extension Version
tried both 2024.3.1 and 2024.4.0
Jupyter logs
Coding Language and Runtime Version
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Language Extension Version (if applicable)
Pylance: tried both 2024.4.1 and 2024.5.1 Python: v2024.6.0
Anaconda Version (if applicable)
N/A, but with Poetry 1.8.3
Running Jupyter locally or remotely?
Remote