Python syntax highlighting fails without window reload

[tfire]

Getting python syntax highlighting to apply to newly written code is requiring a window reload.

• Remote SSH
• Cursor Pyright + ms-python Python
• Amazon Linux 2 (Red Hat)

If I type anything new, eg, print() and save the file, the following is logged:

2024-01-27 21:26:27.146 [error] [ms-python.python] provider FAILED
2024-01-27 21:26:27.146 [error] Error: No semantic tokens provided/needed for file:///home/ec2-user/my/python/file.py
    at /home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2308663
    at re (/home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2308957)
    at /home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2303741
    at Immediate.<anonymous> (/home/ec2-user/.cursor-server/extensions/ms-python.python-2023.14.0-universal/out/client/extension.js:2:2303761)
    at process.processImmediate (node:internal/timers:476:21)

See the syntax highlighting above not applied to the last two prints.

[tfire]

2024-01-27 22:42:27.921 [info] [Error - 10:42:27 PM] Request textDocument/semanticTokens/full/delta failed.

[Sanger2000]

With Cursor Pyright, you should be using anysphere.python (rather than ms-python.python). Can help you figure this out via email if you send details to aman@anysphere.co

[tfire]

Thanks. Where is anysphere.python supposed to come from? It's not mentioned in any docs, across google, or in the extensions search. Taking this over email won't help others lacking the same information.

[arkaydeus]

In addition isort and Black Formatter rely on ms-python (preventing it being uninstalled).

Is there a decent solution here where we can still sort imports and format, while using Cursor Python?

Is there any reason why this hasn't been documented and explained?

[tfire]

I'm guessing anysphere.python is an unallowed fork of ms.python

[lukaemon]

Thanks. Where is anysphere.python supposed to come from? It's not mentioned in any docs, across google, or in the extensions search. Taking this over email won't help others lacking the same information.

Did you find anysphere.python?

[arkaydeus]

Is there a solution to this. Python still looks completely broken in Cursor. One of the most popular languages out there completely unusable in Cursor. Surely there must be a solution?

[arkaydeus]

To summarise, cursor + cursor pyright + python doesn't work as syntax highlighting fails and that's a core part of the language server. It's basically failing to pickup tokens. Something is broken which means you can't really use the Python language in Cursor at the moment.

[arkaydeus]

Great. But please could you explain how you fixed it as it still doesn't work for other people.

[navels]

I would also like to know the magical incantation required to get syntax highlighting working.

[navels]

Nothing at the client side. They pushed something in their backend that solved my problems.

lol it was probably if username == "lukaemon" . . .

[rian-dolphin]

Any update on this?

[jkryanchou]

Any update to this issue ? I have met the same issue. While using the pylance language server won't raise the exception

2024-09-01 12:46:25.111 [info] (Client) Pylance async client (2024.8.1) started with python extension (2024.12.3)
2024-09-01 12:46:25.357 [info] [Info  - 12:46:25 PM] (592) Pylance language server 2024.8.1 (pyright version 1.1.373, commit ee424479) starting
2024-09-01 12:46:25.357 [info] [Info  - 12:46:25 PM] (592) Server root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist
2024-09-01 12:46:25.362 [info] [Info  - 12:46:25 PM] (592) Starting service instance "icc-cloud-audit"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Setting pythonPath for service "icc-cloud-audit": "/workspaces/icc-cloud-audit/.venv/bin/python"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Setting environmentName for service "project": "3.12.5 (.venv venv)"
2024-09-01 12:46:25.386 [info] [Info  - 12:46:25 PM] (592) Loading pyproject.toml file at /workspaces/<project>/pyproject.toml
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Pyproject file "/workspaces/<project>/pyproject.toml" has no "[tool.pyright]" section.
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) No include entries specified; assuming /workspaces/<project>
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/node_modules
2024-09-01 12:46:25.390 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/__pycache__
2024-09-01 12:46:25.391 [info] [Info  - 12:46:25 PM] (592) Auto-excluding **/.*
2024-09-01 12:46:25.416 [info] [Info  - 12:46:25 PM] (592) Assuming Python version 3.12.5.final.0
2024-09-01 12:46:25.632 [info] [Info  - 12:46:25 PM] (592) Found 18 source files
2024-09-01 12:46:25.634 [info] [Info  - 12:46:25 PM] (592) Background analysis(1) root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist
2024-09-01 12:46:25.634 [info] [Info  - 12:46:25 PM] (592) Background analysis(1) started
2024-09-01 12:46:33.402 [info] [Info  - 12:46:33 PM] (592) Indexer background runner(2) root directory: file:///root/.cursor-server/extensions/ms-python.vscode-pylance-2024.8.1/dist (index)
2024-09-01 12:46:33.402 [info] [Info  - 12:46:33 PM] (592) Indexing(2) started
2024-09-01 12:46:33.939 [info] [Info  - 12:46:33 PM] (592) scanned(2) 197 files over 1 exec env
2024-09-01 12:46:34.212 [info] [Info  - 12:46:34 PM] (592) indexed(2) 149 files over 1 exec env
2024-09-01 12:46:34.288 [info] [Info  - 12:46:34 PM] (592) Indexing finished(2).

While using the pyright raise exception as unexpected.

2024-09-01 13:06:27.722 [info] [Info  - 1:06:27 PM] Pyright language server 1.1.325 starting
2024-09-01 13:06:27.722 [info] [Info  - 1:06:27 PM] Server root directory: /root/.cursor-server/extensions/anysphere.pyright-1.1.327-universal/dist
2024-09-01 13:06:27.724 [info] [Info  - 1:06:27 PM] Starting service instance "x-insight"
2024-09-01 13:06:27.745 [info] [Info  - 1:06:27 PM] Setting pythonPath for service "x-insight": "/venv/bin/python"
2024-09-01 13:06:27.752 [info] [Info  - 1:06:27 PM] Loading pyproject.toml file at /workspaces/x-insight/pyproject.toml
2024-09-01 13:06:27.756 [info] [Info  - 1:06:27 PM] Pyproject file "/workspaces/x-insight/pyproject.toml" has no "[tool.pyright]" section.
2024-09-01 13:06:27.756 [info] [Warn  - 1:06:27 PM] stubPath /workspaces/x-insight/typings is not a valid directory.
2024-09-01 13:06:27.776 [info] [Info  - 1:06:27 PM] Assuming Python version 3.12
2024-09-01 13:06:27.875 [info] [Info  - 1:06:27 PM] Auto-excluding /workspaces/x-insight/venv3
2024-09-01 13:06:27.884 [info] [Info  - 1:06:27 PM] Found 61 source files
2024-09-01 13:06:53.425 [info] [Error - 1:06:53 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:53.425 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:54.571 [info] [Error - 1:06:54 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:54.571 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:57.776 [info] [Error - 1:06:57 PM] Request textDocument/semanticTokens/full/delta failed.
2024-09-01 13:06:57.776 [info]   Message: No semantic tokens provided/needed for file:///workspaces/x-insight/xinsight/telegram_bot.py
  Code: -32603 
2024-09-01 13:06:59.416 [info] [Error - 1:06:59 PM] Request textDocument/semanticTokens/full/delta failed

[rian-dolphin]

I seem to have got it working though I'm not quite sure what did it. I did the following:

Uninstalled cursor by doing the following (on mac):

1. Quit Cursor if it's running.
2. Go to the Applications folder and move the Cursor app to the Trash.
3. Clean up associated files:
   • Delete any folders or files related to Cursor in these directories: ~/Library/Application Support/ ~/Library/Caches/ ~/Library/Preferences/ ~/Library/Saved Application State/
4. Empty the Trash.

Reinstalled cursor and selected the option to continue from scratch without any of my vscode settings.

Then the highlighting was working, BUT when I changed to the Dark Modern theme it stopped working again. I then switched back to Anysphere Dark and back to Dark Modern again and it was working.

As I said, I'm not sure what did it but hopefully it can help someone else.

[arkaydeus]

I'm STILL getting this issue when using the Dark+ theme.

How to recreate:

Fresh install of cursor. Open a Python project Add new import (not highlighted) Use new import (not highlighted)

Note that if you close the file and open it again, it then correctly highlights:

This is incredibly frustrating and has lasted over 6 months now. PLEASE can someone provide advice on how they were able to solve this?

[arkaydeus]

BUT when I changed to the Dark Modern theme it stopped working again.

Can you confirm if you got it working persistently? For me when I restart, it correctly highlights when the file is first open or the window is refreshed, but it doesn't carry on working when I type new code. It looks like the deltas aren't being passed to the language server.

[arkaydeus]

OK, I've made some progress on this.

Currently running MacOS. I installed a new profile (user) on my machine and strangely all worked fine. What I noticed though, is that different versions of the extensions are installed.

Compare the two. The darwin-arm64 versions work fine. The universal versions don't and keep running into issues. They're also older.

Following up and checking the two extensions.json files you can see the working version on the right and the broken version on the left.

On the right the targetPlatform is ‘darwin-arm64’ which is correct. On the left it’s ‘universal’ and then the version is 2024.5.1 and not 2024.12.3

This is incredibly frustrating. Same machine. Same OS. Just different user profile.

Would be really interested to see how I can force a particular architecture for extensions.