Support for Function return type hinting

[haelmj]

Environment data

• Language Server version: v2022.2.1
• OS and version: macOS Big Sur Version 11.4
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.2

Expected behaviour

When return type hint is added to function definition, syntax highlighting should remain consistent.

Actual behaviour

When return type hint is added to function definition, the syntax highlighting for the next function is disabled

Logs

(6698) Background analysis message: setFileOpened
(6698) Background analysis message: markFilesDirty
(6698) [FG] parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (2ms)
(6698) [FG] binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (0ms)
(6698) Background analysis message: analyze
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py ...
(6698) [BG(1)]   parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (1ms)
(6698) [BG(1)]   binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (1ms)
(6698) [BG(1)]   checking: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (24ms)
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (26ms)
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: resumeAnalysis
(6698) [BG(1)] indexing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py [found 6] (1ms)
(6698) Indexing Done: /Users/haelmj/Documents/Security/blackhatpython/netcat.py
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getSemanticTokens delta
(6698) [BG(1)] getSemanticTokens delta previousResultId:1644737436849 at /Users/haelmj/Documents/Security/blackhatpython/netcat.py (5ms)
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: setFileOpened
(6698) Background analysis message: markFilesDirty
[Info  - 08:30:45] (6698) SourceFile: Received fs event 'change' for path '/Users/haelmj/Documents/Security/blackhatpython/netcat.py'
(6698) Background analysis message: markFilesDirty
(6698) [FG] parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (1ms)
(6698) [FG] binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (0ms)
(6698) Background analysis message: analyze
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py ...
(6698) [BG(1)]   parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (2ms)
(6698) [BG(1)]   binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (0ms)
(6698) [BG(1)]   checking: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (17ms)
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (19ms)
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: resumeAnalysis
(6698) [BG(1)] indexing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py [found 6] (0ms)
(6698) Indexing Done: /Users/haelmj/Documents/Security/blackhatpython/netcat.py
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getSemanticTokens delta
(6698) [BG(1)] getSemanticTokens delta previousResultId:1644737444929 at /Users/haelmj/Documents/Security/blackhatpython/netcat.py (4ms)
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: setFileOpened
(6698) Background analysis message: markFilesDirty
(6698) [FG] parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (4ms)
(6698) [FG] binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (1ms)
(6698) Background analysis message: analyze
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py ...
(6698) [BG(1)]   parsing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (3ms)
(6698) [BG(1)]   binding: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (1ms)
(6698) [BG(1)]   checking: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (27ms)
(6698) [BG(1)] analyzing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py (31ms)
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: resumeAnalysis
(6698) [BG(1)] indexing: /Users/haelmj/Documents/Security/blackhatpython/netcat.py [found 6] (1ms)
(6698) Indexing Done: /Users/haelmj/Documents/Security/blackhatpython/netcat.py
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getSemanticTokens delta
(6698) [BG(1)] getSemanticTokens delta previousResultId:1644737446176 at /Users/haelmj/Documents/Security/blackhatpython/netcat.py (6ms)
[Info  - 08:30:47] (6698) SourceFile: Received fs event 'change' for path '/Users/haelmj/Documents/Security/blackhatpython/netcat.py'
[Info  - 08:30:47] (6698) SourceFile: Received fs event 'change' for path '/Users/haelmj/Documents/Security/blackhatpython/.dccache'
(6698) Background analysis message: markFilesDirty
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: getDiagnosticsForRange
(6698) Background analysis message: analyze
[Info  - 08:30:48] (6698) SourceFile: Received fs event 'change' for path '/Users/haelmj/Documents/Security/blackhatpython/netcat.py'
[Info  - 08:30:48] (6698) SourceFile: Received fs event 'change' for path '/Users/haelmj/Documents/Security/blackhatpython/.dccache'
(6698) Background analysis message: markFilesDirty
(6698) Background analysis message: analyze

Code Snippet / Additional information

In the following the second class method below, syntax highlighting on def will be disabled due to the return type hint None

class NetCat:
    def __init__(self, args, buffer=None) -> None:
        self.args = args
        self.buffer = buffer
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    def listen(self):
        self.socket.bind((self.args.target, self.args.port))
        self.socket.listen(5)

[erictraut]

I'm not seeing a difference between the two. Here's what I see when I add/remove the -> None return type annotation. If you're seeing something different, perhaps you could post a screen shot?

[haelmj]

@erictraut That's odd, notice the change in syntax highlighting in the image below. If I take None, out it works fine but changes once I put it in

[erictraut]

Strange. Let's see if someone else on the Pylance team can reproduce what you're seeing.

I have a partial theory about what might be happening here. Pyright, the type checker that Pylance is built upon, has some special techniques for dealing with classes whose constructors are completely unannotated. In such cases, it internally treats that class as though it's generic with each of the constructor input parameters representing a different type parameter. This improves type inference. For example, if you construct a new object nc = NetCat([1, 2], ""), pyright will treat that as though it's a generic class instance with the type NetCat[list[int], str], and if you then access nc.args, it will infer the type to be list[int]. As soon as pyright sees a type annotation (even a return type annotation) on the __init__ method, it disables this technique, so accessing nc.args will have an Any type.

If my theory is on track, you should be able to repro the problem by adding a type annotation to any one of the __init__ parameters. For example, if you add a : int after args.

[haelmj]

@erictraut So it seems, this isn't solely for classes but independent functions as well. Apparently, adding just parameter type annotations works just fine until a return type annotation is added in. I've attached some extra screenshots to show what I'm referring to.

It also seems to affect the syntax highlighting for strings within the same code block

[erictraut]

Thanks for the additional experimentation. That disproves my theory.

Let's see if someone else on the Pylance team is able to replicate the behavior you're seeing.

[debonte]

I wasn't able to repro this.

[haelmj]

@debonte Could you share a screenshot of what you did and the output you got?

I have further tested this on a Windows machine with the same result. That verifies that the issue isn't OS-related.

[bschnurr]

couldn't repo.

"Python Language Server" logs would help. https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue

[erictraut]

@haelmj, which version of VS Code are you using? Is it the latest (or at least a recent) version?

[haelmj]

@erictraut I've just tested this on a windows 10 and couldn't repro the issue VSCode version: 1.64.2(user setup) OS: Windows_NT x64 10.0.19044

However, the issue still appears when I test it on MacOS: VSCode version: 1.64.2(Universal) OS: Darwin arm64 20.5.0

[haelmj]

@bschnurr I'll attempt the troubleshooting over the weekend and let you know how that goes.

[haelmj]

@erictraut I verified that I have the latest versions of VSCode as well as Pylance. However, the behaviour still persists on the mac

[haelmj]

@bschnurr Here are the logs:

[Info  - 6:33:54 PM] (1838) Pylance language server 2022.2.4 (pyright 101e717a) starting
[Info  - 6:33:54 PM] (1838) Server root directory: /Users/haelmj/.vscode/extensions/ms-python.vscode-pylance-2022.2.4/dist
[Info  - 6:33:54 PM] (1838) No configuration file found.
[Info  - 6:33:54 PM] (1838) No pyproject.toml file found.
[Info  - 6:33:54 PM] (1838) Setting pythonPath for service "blackhatpython": "/Users/haelmj/Documents/Security/blackhatpython/venv3/bin/python"
[Warn  - 6:33:54 PM] (1838) stubPath /Users/haelmj/Documents/Security/blackhatpython/typings is not a valid directory.
[Info  - 6:33:54 PM] (1838) Assuming Python version 3.8
[Info  - 6:33:54 PM] (1838) Assuming Python platform Darwin
[Info  - 6:33:54 PM] (1838) Searching for source files
[Info  - 6:33:54 PM] (1838) Auto-excluding /Users/haelmj/Documents/Security/blackhatpython/venv3
[Info  - 6:33:54 PM] (1838) Found 4 source files
[Info  - 6:33:54 PM] (1838) Background analysis(1) root directory: /Users/haelmj/.vscode/extensions/ms-python.vscode-pylance-2022.2.4/dist
[Info  - 6:33:54 PM] (1838) Background analysis(1) started
[Info  - 6:33:54 PM] (1838) Indexer background runner(2) root directory: /Users/haelmj/.vscode/extensions/ms-python.vscode-pylance-2022.2.4/dist (index)
[Info  - 6:33:54 PM] (1838) Indexing(2) started
[Info  - 6:33:55 PM] (1838) Indexer done(2). indexed 424 files

[bschnurr]

https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue

Please: Enable trace logging by adding "python.analysis.logLevel": "Trace" to your settings.json configuration file.

Adding this will cause a large amount of info to be printed to the Python output panel. This should not be left long term, as the performance impact of the logging is significant.

Select "View: Toggle Output" from the command palette (Ctrl+Shift+P on Windows/Linux, Command+Shift+P on macOS), then select "Python Language Server" in the dropdown on the right.

[judej]

Please reopen this issue with additional logging as described if this is still an issue.