Python: Executing from the console occasionally hangs

[seeM]

When this happens, pressing enter in the console does nothing. The console does not show a "stop" symbol to indicate that it's busy, nor does a new prompt appear.

Pressing enter multiple times also does nothing at that point.

Once the hang ends, multiple executions are sent, one for each time the user pressed enter. All of them are displayed in the console.

---

From the logs, it seems that the kernel becomes non-responsive. Positron sends an is_complete_request, but does not receive a status(busy) (which allows us to queue more executions), nor does it receive an is_complete_reply.

I also noticed that just before the first is_complete_request, the LSP received a textDocument/hover request and began processing it. It loads a bunch of pickle files thereafter. I wonder if we're trying to do too much in the same thread/process?

[seeM]

This is definitely an issue with the runtime, but would we also want the Positron Console to be more robust to a problematic runtime? Specifically, do we want to block further executions if we're waiting for the is_complete_reply, even though we haven't yet received a status(busy)?

[seeM]

I seem to be able to reliably cause a hang by:

• Opening a new workspace (cmd+shift+n)
• Opening a new file (cmd+n), saving it (cmd+s)
• Typing the single line from fastai.vision.all import *
• Pressing cmd+enter on that line. Pressing it any amount of times at this point will not cause a hang
• Moving the cursor in the document
• Pressing cmd+enter again

I don't seem to be able to reproduce the issue by directly executing from the console. Only cmd+enter causes a hang.

Here are the logs immediately following the above:

[Python] [pygls.protocol] Received b'Content-Length: 255\r\n\r\n{"jsonrpc":"2.0","id":3,"method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///Users/seem/test_kernel_hang.py"},"range":{"start":{"line":0,"character":30},"end":{"line":0,"character":30}},"context":{"diagnostics":[],"triggerKind":2}}}'
[Python] [pygls.protocol] Request message received.
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/all.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/basics.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/all.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/torch_basics.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/imports.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/all.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/imports.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/foundation.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/basics.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/dispatch.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/utils.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/xtras.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/parallel.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/meta.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/test.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/net.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/transform.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastcore/script.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/torch_imports.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/torch_core.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/layers.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/losses.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/core.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/load.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/external.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/transforms.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/data/block.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/optimizer.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/core.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/learner.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/metrics.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/interpret.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/tabular/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/tabular/core.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/all.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/data.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/fp16.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/progress.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/hook.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/mixup.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/schedule.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/tracker.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/rnn.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/training.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/preds.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/callback/channelslast.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/augment.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/core.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/data.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/models/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/models/all.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/models/xresnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/models/unet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/models/tvm.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/__init__.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/alexnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/convnext.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/densenet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/efficientnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/googlenet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/inception.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/mnasnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/mobilenet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/mobilenetv2.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/mobilenetv3.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/regnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/resnet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/shufflenetv2.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/squeezenet.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/vgg.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/vision_transformer.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/swin_transformer.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/torchvision/models/maxvit.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/learner.py
[Python] [parso.cache] pickle loaded: /Users/seem/Library/Python/3.9/lib/python/site-packages/fastai/vision/utils.py

[seeM]

It turns out this was not the only issue. @jjallaire found another hang which you can repro as follows:

1. Start with a blank document
2. Type from fastai.vision.all import Im into the document. The kernel will hang
3. Try to execute anything in the console and it will get stuck at is_complete_request

[seeM]

Looks like the issue has been logged on Jedi (https://github.com/davidhalter/jedi/issues/1721) and jedi-language-server (https://github.com/pappasam/jedi-language-server/issues/57)! I implemented the same workaround in https://github.com/posit-dev/positron-python/pull/158.

---

While investing this, I realised that jedi-language-server runs each request synchronously, which I think would block our kernel. I tried using pygls' (the underlying language server framework) threaded option, and it does seem to improve things a little, but it causes the shutdown to become much slower.

I'm going to leave things as is for now, but I suspect we may have to deal with more performance issues related to this in future.

[seeM]

This is ready for review on main. Both the first and second test cases above should be fixed

[isabelizimm]

Was able to execute from fastai.vision.all import * and from fastai.vision.all import Image (got an error trying to import Im) in new files via cmd+enter