Syntax highlighting wrong in `for` loop with generator

[jakelevi1996]

Type: Bug

Behaviour

Syntax highlighting is wrong in generator expressions, specifically the in changes from purple to yellow (default dark+ colour theme) when the object being iterated over is a defined by a function with a yield expression.

Steps to reproduce:

Enter the following code in a Python file:

def g(x):
    for i in x:
        yield x

for i in range(23):
    pass

for i in g(range(23)):
    pass

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` XXX ```

Extension version: 2024.4.1 VS Code version: Code 1.87.2 (863d2581ecda6849923a2118d93a088b0745d9d6, 2024-03-08T15:14:59.643Z) OS version: Linux x64 6.1.0-1034-oem snap Modes:

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.12
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

User Settings

``` languageServer: "Pylance" testing • pytestArgs: "" • pytestEnabled: true ```

Installed Extensions

|Extension Name|Extension Id|Version| |---|---|---| |C/C++|ms-vscode.cpptools|1.19.9| |Fix JSON|oliversturm.fix-json|0.1.2| |Git Graph|mhutchie.git-graph|1.30.0| |GitLens — Git supercharged|eamodio.gitlens|14.9.0| |Hex Editor|ms-vscode.hexeditor|1.9.14| |JavaScript Debugger|ms-vscode.js-debug|1.87.0| |JavaScript Debugger Companion Extension|ms-vscode.js-debug-companion|1.1.2| |LaTeX Workshop|James-Yu.latex-workshop|9.19.2| |Markdown All in One|yzhang.markdown-all-in-one|3.6.2| |Material Icon Theme|PKief.material-icon-theme|4.34.0| |Pylance|ms-python.vscode-pylance|2024.4.1| |Python|ms-python.python|2024.4.0| |Python Debugger|ms-python.debugpy|2024.4.0| |Rainbow CSV|mechatroner.rainbow-csv|3.11.0| |Rewrap|stkb.rewrap|1.16.3| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.8| |transpose|v4run.transpose|0.0.6| |vscode-pdf|tomoki1207.pdf|1.2.2|

System Info

|Item|Value| |---|---| |CPUs|13th Gen Intel(R) Core(TM) i7-1365U (12 x 4863)| |GPU Status|2d_canvas: unavailable_software
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: disabled_software
multiple_raster_threads: enabled_on
opengl: disabled_off
rasterization: disabled_software
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: disabled_software
video_encode: disabled_software
vulkan: disabled_off
webgl: unavailable_software
webgl2: unavailable_software
webgpu: disabled_off| |Load (avg)|1, 1, 1| |Memory (System)|15.06GB (5.26GB free)| |Process Argv|--no-sandbox --force-user-env --unity-launch --crash-reporter-id 6f80bf97-b84e-4222-a95e-8a99afa32ff5| |Screen Reader|no| |VM|0%| |DESKTOP_SESSION|ubuntu| |XDG_CURRENT_DESKTOP|Unity| |XDG_SESSION_DESKTOP|ubuntu| |XDG_SESSION_TYPE|wayland|

A/B Experiments

``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vstes627:30244334 vscoreces:30445986 vscod805cf:30301675 binariesv615:30325510 vsaa593:30376534 py29gd2263:30899288 c4g48928:30535728 azure-dev_surveyone:30548225 vscrp:30673768 2i9eh265:30646982 962ge761:30959799 pythongtdpath:30769146 welcomedialogc:30910334 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pyreplss1:30897532 pythonmypyd1:30879173 pythoncet0:30885854 h48ei257:31000450 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 d6fi2104:30959263 cppperfnew:31000557 d34g3935:30971562 fegfb526:30981948 bg6jg535:30979843 ccp2r3:30993541 dsvsc020:30976470 pythonait:31006305 jchc7451:30973076 gee8j676:31009558 dsvsc021:30996838 gd77d436:30999572 vstestt:31010172 ```

[jakelevi1996]

Here is an example screenshot:

[debonte]

Dupe of https://github.com/microsoft/pylance-release/issues/4845?

[StellaHuang95]

yes that should be a dupe. Debugging this, I can see the overridden token modifier being added to the last in. Closing as dupe. Please follow the linked issue above for updates. Thanks!