pygls 1.0.0a0

[dimbleby]

pygls is going 1.0, and the main change is that they're migrating away from their homegrown pydantic models for the protocol and using https://pypi.org/project/lsprotocol/ (https://github.com/microsoft/lsprotocol) - which seems to be autogenerated (and uses cattrs).

that turns out not to be a tremendously impactful change: a few things get renamed in more-or-less obvious ways, the validation gets slightly stricter in a couple of places, that's about it.

I don't expect you'll want to merge / publish before they actually go 1.0.0. But if and when you're happy that this seems anyway not worse than before, I expect they'd be glad to hear about it in https://github.com/openlawlibrary/pygls/pull/273

[pappasam]

Thanks for the PR! Tried testing it with my current client (coc.nvim) and it breaks with the following deserialization message:

ERROR:pygls.protocol:Unable to deserialize message
  + Exception Group Traceback (most recent call last):
  |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/pygls/protocol.py", line 347, in _deserialize_message
  |     return converter.structure(data, request_type)
  |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 309, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure lsprotocol.types.InitializeRequest>", line 26, in structure_InitializeRequest
  |     if errors: raise __c_cve('While structuring ' + 'InitializeRequest', errors, __cl)
  | cattrs.errors.ClassValidationError: While structuring InitializeRequest (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    |   File "<cattrs generated structure lsprotocol.types.InitializeRequest>", line 10, in structure_InitializeRequest
    |     res['params'] = __c_structure_params(o['params'], __c_type_params)
    |   File "<cattrs generated structure lsprotocol.types.InitializeParams>", line 63, in structure_InitializeParams
    |     if errors: raise __c_cve('While structuring ' + 'InitializeParams', errors, __cl)
    | cattrs.errors.ClassValidationError: While structuring InitializeParams (1 sub-exception)
    | Structuring class InitializeRequest @ attribute params
    +-+---------------- 1 ----------------
      | Exception Group Traceback (most recent call last):
      |   File "<cattrs generated structure lsprotocol.types.InitializeParams>", line 5, in structure_InitializeParams
      |     res['capabilities'] = __c_structure_capabilities(o['capabilities'], __c_type_capabilities)
      |   File "<cattrs generated structure lsprotocol.types.ClientCapabilities>", line 40, in structure_ClientCapabilities
      |     if errors: raise __c_cve('While structuring ' + 'ClientCapabilities', errors, __cl)
      | cattrs.errors.ClassValidationError: While structuring ClientCapabilities (1 sub-exception)
      | Structuring class InitializeParams @ attribute capabilities
      +-+---------------- 1 ----------------
        | Exception Group Traceback (most recent call last):
        |   File "<cattrs generated structure lsprotocol.types.ClientCapabilities>", line 12, in structure_ClientCapabilities
        |     res['text_document'] = __c_structure_text_document(o['textDocument'], __c_type_text_document)
        |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 574, in _structure_optional
        |     return self._structure_func.dispatch(other)(obj, other)
        |   File "<cattrs generated structure lsprotocol.types.TextDocumentClientCapabilities>", line 184, in structure_TextDocumentClientCapabilities
        |     if errors: raise __c_cve('While structuring ' + 'TextDocumentClientCapabilities', errors, __cl)
        | cattrs.errors.ClassValidationError: While structuring TextDocumentClientCapabilities (1 sub-exception)
        | Structuring class ClientCapabilities @ attribute text_document
        +-+---------------- 1 ----------------
          | Exception Group Traceback (most recent call last):
          |   File "<cattrs generated structure lsprotocol.types.TextDocumentClientCapabilities>", line 144, in structure_TextDocumentClientCapabilities
          |     res['semantic_tokens'] = __c_structure_semantic_tokens(o['semanticTokens'], __c_type_semantic_tokens)
          |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 574, in _structure_optional
          |     return self._structure_func.dispatch(other)(obj, other)
          |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilities>", line 54, in structure_SemanticTokensClientCapabilities
          |     if errors: raise __c_cve('While structuring ' + 'SemanticTokensClientCapabilities', errors, __cl)
          | cattrs.errors.ClassValidationError: While structuring SemanticTokensClientCapabilities (1 sub-exception)
          | Structuring class TextDocumentClientCapabilities @ attribute semantic_tokens
          +-+---------------- 1 ----------------
            | Exception Group Traceback (most recent call last):
            |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilities>", line 5, in structure_SemanticTokensClientCapabilities
            |     res['requests'] = __c_structure_requests(o['requests'], __c_type_requests)
            |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilitiesRequestsType>", line 16, in structure_SemanticTokensClientCapabilitiesRequestsType
            |     if errors: raise __c_cve('While structuring ' + 'SemanticTokensClientCapabilitiesRequestsType', errors, __cl)
            | cattrs.errors.ClassValidationError: While structuring SemanticTokensClientCapabilitiesRequestsType (1 sub-exception)
            | Structuring class SemanticTokensClientCapabilities @ attribute requests
            +-+---------------- 1 ----------------
              | Traceback (most recent call last):
              |   File "<cattrs generated structure lsprotocol.types.SemanticTokensClientCapabilitiesRequestsType>", line 12, in structure_SemanticTokensClientCapabilitiesRequestsType
              |     res['full'] = __c_structure_full(o['full'], __c_type_full)
              |   File "/home/sroeca/src/Personal/jedi-language-server/.venv/lib/python3.10/site-packages/cattrs/converters.py", line 377, in _structure_error
              |     raise StructureHandlerNotFoundError(msg, type_=cl)
              | cattrs.errors.StructureHandlerNotFoundError: Unsupported type: typing.Union[bool, lsprotocol.types.SemanticTokensClientCapabilitiesRequestsTypeFullType1, NoneType]. Register a structure hook for it.
              | Structuring class SemanticTokensClientCapabilitiesRequestsType @ attribute full
              +------------------------------------

Validation of this breaks:

[Trace - 12:05:09 PM] Sending request 'initialize - (0)'.
Params: {
    "processId": 1105947,
    "rootPath": "/home/sroeca/src/Personal/jedi-language-server",
    "rootUri": "file:///home/sroeca/src/Personal/jedi-language-server",
    "capabilities": {
        "workspace": {
            "applyEdit": true,
            "workspaceEdit": {
                "documentChanges": true,
                "resourceOperations": [
                    "create",
                    "rename",
                    "delete"
                ],
                "failureHandling": "undo",
                "normalizesLineEndings": true,
                "changeAnnotationSupport": {
                    "groupsOnLabel": false
                }
            },
            "didChangeConfiguration": {
                "dynamicRegistration": true
            },
            "didChangeWatchedFiles": {
                "dynamicRegistration": true,
                "relativePatternSupport": true
            },
            "codeLens": {
                "refreshSupport": true
            },
            "executeCommand": {
                "dynamicRegistration": true
            },
            "configuration": true,
            "fileOperations": {
                "dynamicRegistration": true,
                "didCreate": true,
                "didRename": true,
                "didDelete": true,
                "willCreate": true,
                "willRename": true,
                "willDelete": true
            },
            "semanticTokens": {
                "refreshSupport": true
            },
            "inlayHint": {
                "refreshSupport": true
            },
            "inlineValue": {
                "refreshSupport": true
            },
            "diagnostics": {
                "refreshSupport": true
            },
            "symbol": {
                "dynamicRegistration": true,
                "symbolKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25,
                        26
                    ]
                },
                "tagSupport": {
                    "valueSet": [
                        1
                    ]
                },
                "resolveSupport": {
                    "properties": [
                        "location.range"
                    ]
                }
            },
            "workspaceFolders": true
        },
        "textDocument": {
            "publishDiagnostics": {
                "relatedInformation": true,
                "versionSupport": true,
                "tagSupport": {
                    "valueSet": [
                        1,
                        2
                    ]
                },
                "codeDescriptionSupport": true,
                "dataSupport": true
            },
            "synchronization": {
                "dynamicRegistration": true,
                "willSave": true,
                "willSaveWaitUntil": true,
                "didSave": true
            },
            "completion": {
                "dynamicRegistration": true,
                "contextSupport": true,
                "completionItem": {
                    "snippetSupport": true,
                    "commitCharactersSupport": true,
                    "documentationFormat": [
                        "markdown",
                        "plaintext"
                    ],
                    "deprecatedSupport": true,
                    "preselectSupport": true,
                    "insertReplaceSupport": true,
                    "tagSupport": {
                        "valueSet": [
                            1
                        ]
                    },
                    "resolveSupport": {
                        "properties": [
                            "documentation",
                            "detail",
                            "additionalTextEdits"
                        ]
                    },
                    "labelDetailsSupport": true,
                    "insertTextModeSupport": {
                        "valueSet": [
                            1,
                            2
                        ]
                    }
                },
                "completionItemKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25
                    ]
                },
                "insertTextMode": 2,
                "completionList": {
                    "itemDefaults": [
                        "commitCharacters",
                        "editRange",
                        "insertTextFormat",
                        "insertTextMode"
                    ]
                }
            },
            "hover": {
                "dynamicRegistration": true,
                "contentFormat": [
                    "markdown",
                    "plaintext"
                ]
            },
            "signatureHelp": {
                "dynamicRegistration": true,
                "contextSupport": true,
                "signatureInformation": {
                    "documentationFormat": [
                        "markdown",
                        "plaintext"
                    ],
                    "activeParameterSupport": true,
                    "parameterInformation": {
                        "labelOffsetSupport": true
                    }
                }
            },
            "references": {
                "dynamicRegistration": true
            },
            "definition": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "documentHighlight": {
                "dynamicRegistration": true
            },
            "documentSymbol": {
                "dynamicRegistration": true,
                "symbolKind": {
                    "valueSet": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        9,
                        10,
                        11,
                        12,
                        13,
                        14,
                        15,
                        16,
                        17,
                        18,
                        19,
                        20,
                        21,
                        22,
                        23,
                        24,
                        25,
                        26
                    ]
                },
                "hierarchicalDocumentSymbolSupport": true,
                "tagSupport": {
                    "valueSet": [
                        1
                    ]
                },
                "labelSupport": true
            },
            "codeAction": {
                "dynamicRegistration": true,
                "isPreferredSupport": true,
                "disabledSupport": true,
                "dataSupport": true,
                "honorsChangeAnnotations": false,
                "resolveSupport": {
                    "properties": [
                        "edit"
                    ]
                },
                "codeActionLiteralSupport": {
                    "codeActionKind": {
                        "valueSet": [
                            "",
                            "quickfix",
                            "refactor",
                            "refactor.extract",
                            "refactor.inline",
                            "refactor.rewrite",
                            "source",
                            "source.organizeImports"
                        ]
                    }
                }
            },
            "codeLens": {
                "dynamicRegistration": true
            },
            "formatting": {
                "dynamicRegistration": true
            },
            "rangeFormatting": {
                "dynamicRegistration": true
            },
            "onTypeFormatting": {
                "dynamicRegistration": true
            },
            "rename": {
                "dynamicRegistration": true,
                "prepareSupport": true,
                "honorsChangeAnnotations": true,
                "prepareSupportDefaultBehavior": 1
            },
            "documentLink": {
                "dynamicRegistration": true,
                "tooltipSupport": true
            },
            "typeDefinition": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "implementation": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "declaration": {
                "dynamicRegistration": true,
                "linkSupport": true
            },
            "colorProvider": {
                "dynamicRegistration": true
            },
            "foldingRange": {
                "dynamicRegistration": true,
                "rangeLimit": 5000,
                "lineFoldingOnly": true,
                "foldingRangeKind": {
                    "valueSet": [
                        "comment",
                        "imports",
                        "region"
                    ]
                },
                "foldingRange": {
                    "collapsedText": false
                }
            },
            "selectionRange": {
                "dynamicRegistration": true
            },
            "callHierarchy": {
                "dynamicRegistration": true
            },
            "linkedEditingRange": {
                "dynamicRegistration": true
            },
            "semanticTokens": {
                "dynamicRegistration": true,
                "tokenTypes": [
                    "namespace",
                    "type",
                    "class",
                    "enum",
                    "interface",
                    "struct",
                    "typeParameter",
                    "parameter",
                    "variable",
                    "property",
                    "enumMember",
                    "event",
                    "function",
                    "method",
                    "macro",
                    "keyword",
                    "modifier",
                    "comment",
                    "string",
                    "number",
                    "regexp",
                    "decorator",
                    "operator"
                ],
                "tokenModifiers": [
                    "declaration",
                    "definition",
                    "readonly",
                    "static",
                    "deprecated",
                    "abstract",
                    "async",
                    "modification",
                    "documentation",
                    "defaultLibrary"
                ],
                "formats": [
                    "relative"
                ],
                "requests": {
                    "range": true,
                    "full": {
                        "delta": true
                    }
                },
                "multilineTokenSupport": false,
                "overlappingTokenSupport": false,
                "serverCancelSupport": true,
                "augmentsSyntaxTokens": true
            },
            "inlayHint": {
                "dynamicRegistration": true,
                "resolveSupport": {
                    "properties": [
                        "tooltip",
                        "textEdits",
                        "label.tooltip",
                        "label.location",
                        "label.command"
                    ]
                }
            },
            "inlineValue": {
                "dynamicRegistration": true
            },
            "diagnostic": {
                "dynamicRegistration": true,
                "relatedDocumentSupport": true
            },
            "typeHierarchy": {
                "dynamicRegistration": true
            }
        },
        "window": {
            "showMessage": {
                "messageActionItem": {
                    "additionalPropertiesSupport": true
                }
            },
            "showDocument": {
                "support": true
            },
            "workDoneProgress": true
        },
        "general": {
            "regularExpressions": {
                "engine": "ECMAScript",
                "version": "ES2020"
            },
            "markdown": {
                "parser": "marked",
                "version": "4.0.10"
            },
            "positionEncodings": [
                "utf-16"
            ],
            "staleRequestSupport": {
                "cancel": true,
                "retryOnContentModified": [
                    "textDocument/inlayHint",
                    "textDocument/semanticTokens/full",
                    "textDocument/semanticTokens/range",
                    "textDocument/semanticTokens/full/delta"
                ]
            }
        }
    },
    "initializationOptions": {
        "enable": true,
        "startupMessage": false,
        "trace": {
            "server": "verbose"
        },
        "jediSettings": {
            "autoImportModules": [
                "pygls"
            ],
            "caseInsensitiveCompletion": true,
            "debug": false
        },
        "executable": {
            "args": [],
            "command": "jedi-language-server"
        },
        "codeAction": {
            "nameExtractFunction": "jls_extract_def",
            "nameExtractVariable": "jls_extract_var"
        },
        "completion": {
            "disableSnippets": false,
            "resolveEagerly": false,
            "ignorePatterns": []
        },
        "diagnostics": {
            "enable": true,
            "didOpen": true,
            "didChange": true,
            "didSave": true
        },
        "hover": {
            "enable": true,
            "disable": {
                "class": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "function": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "instance": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "keyword": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "module": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "param": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "path": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "property": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                },
                "statement": {
                    "all": false,
                    "names": [],
                    "fullNames": []
                }
            }
        },
        "workspace": {
            "extraPaths": [],
            "symbols": {
                "maxSymbols": 20,
                "ignoreFolders": [
                    ".nox",
                    ".tox",
                    ".venv",
                    "__pycache__",
                    "venv"
                ]
            }
        }
    },
    "trace": "verbose",
    "workspaceFolders": [
        {
            "uri": "file:///home/sroeca/src/Personal/jedi-language-server",
            "name": "jedi-language-server"
        }
    ],
    "locale": "en_US",
    "clientInfo": {
        "name": "coc.nvim",
        "version": "0.0.82"
    }
}

[pappasam]

Based on the spec, it doesn't look like there's anything obviously wrong with the request: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#semanticTokensClientCapabilities

Relevant section:

"semanticTokens": {
    "requests": {
        "range": true,
        "full": {
            "delta": true
        }
    }
}

[karthiknadig]

This looks like a missed cattrs hook for this particular case in lsprotocol. Created https://github.com/microsoft/lsprotocol/issues/95 to track it.

[karthiknadig]

I just published https://pypi.org/project/lsprotocol/2022.0.0a6/ with a fix for the above issue.

@pappasam for context lsprotocol is a package that is generated from the LSP spec. It is currently in alpha, and attempts to take the drudgery out of writing out types and constraints for LSP. Since this is auto-generated there are going to be some issues especially with cattrs hooks for some of the special cases in LSP. If you run into any bugs please file them on https://github.com/microsoft/lsprotocol/issues , with the error stack like from above.

[pappasam]

It works for me now! I'll play with it some more and monitor until the 1.0 release, thanks @dimbleby and @karthiknadig

[dimbleby]

I manually added a py.typed to my local lsprotocol to see the effect, and sure enough mypy offered some errors. I've pushed a commit that resolves them.

(The analogous changes could be made on main: they should still be correct, even if the pygls typing is not good enough to tell us that)

[dimbleby]

@pappasam pygls has now gone 1.0.0, and I've updated accordingly - so far as I know this is good to go.

They snuck in a last-minute change to make the name and version both mandatory on the LanguageServer, that required a small refactor. Otherwise not much is different here since you last looked at it.

lsprotocol didn't yet declare itself non-alpha - there was some discussion (from here) about whether to wait for it but the view seems to be that it's ready enough and will likely stop calling itself alpha shortly. I don't think this project needs to wait for that, but it wouldn't be crazy to do so.

[pappasam]

Note: I migrated to cattrs locally, but the error messages were not very user friendly. I'm just going to keep pydantic for now.