Not compatible with Python 3.11 and pygls 1.0.2

[sylvainfaivre]

I am trying to run salt-lsp with Python 3.11, which is shipped with Ubuntu 23.04.

I first updated the dependancies by running poetry update before poetry install.

I then ran poetry run dump_state_name_completions which run successfully.

But the server failed with the following error :

$ poetry run salt_lsp_server --tcp
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/sylvain/git/salt-lsp/salt_lsp/__main__.py", line 7, in <module>
    from salt_lsp.server import SaltServer, setup_salt_server_capabilities
  File "/home/sylvain/git/salt-lsp/salt_lsp/server.py", line 10, in <module>
    from pygls.lsp import types
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/__init__.py", line 23, in <module>
    from pygls.lsp.types import *
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/types/__init__.py", line 6, in <module>
    from pygls.lsp.types.general_messages import *
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/types/general_messages.py", line 35, in <module>
    from pygls.lsp.types.language_features import (CallHierarchyClientCapabilities,
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/types/language_features/__init__.py", line 2, in <module>
    from pygls.lsp.types.language_features.call_hierarchy import *
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/types/language_features/call_hierarchy.py", line 70, in <module>
    class CallHierarchyIncomingCall(Model):
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pydantic/main.py", line 369, in __new__
    cls.__signature__ = ClassAttribute('__signature__', generate_model_signature(cls.__init__, fields, config))
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pydantic/utils.py", line 231, in generate_model_signature
    merged_params[param_name] = Parameter(
                                ^^^^^^^^^^
  File "/usr/lib/python3.11/inspect.py", line 2722, in __init__
    raise ValueError('{!r} is not a valid parameter name'.format(name))
ValueError: 'from' is not a valid parameter name

This seems related to https://github.com/pydantic/pydantic/issues/4500 which is fixed in pydantic v1.10.3 - see https://github.com/pydantic/pydantic/issues/4552

So I tried to add pydantic = ">=1.10.3" in the [tool.poetry.dependencies] section in pyproject.toml

Then poetry update :

Because pygls (0.11.3) depends on pydantic (>=1.7,<1.9)
 and no versions of pygls match >0.11.3,<0.12.0, pygls (>=0.11.3,<0.12.0) requires pydantic (>=1.7,<1.9).
So, because salt-lsp depends on both pygls (^0.11.3) and pydantic (>=1.10.3), version solving failed.

The next step was to replace pygls = "^0.11.3" with pygls = ">=0.11.3"

I redid the whole steps, then I got another error at runtime :

$ poetry run salt_lsp_server --tcp
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/sylvain/git/salt-lsp/salt_lsp/__main__.py", line 7, in <module>
    from salt_lsp.server import SaltServer, setup_salt_server_capabilities
  File "/home/sylvain/git/salt-lsp/salt_lsp/server.py", line 10, in <module>
    from pygls.lsp import types
ImportError: cannot import name 'types' from 'pygls.lsp' (/home/sylvain/.cache/pypoetry/virtualenvs/salt-lsp-lgM61b6e-py3.11/lib/python3.11/site-packages/pygls/lsp/__init__.py)

This seems related to a breaking change in pygls v1.0 : https://pygls.readthedocs.io/en/latest/pages/migrating-to-v1.html

[dcermak]

I started porting salt-lsp to pygls 1.0+ (see https://github.com/dcermak/salt-lsp/pull/285), but sadly never managed to finish that work. Unfortunately this project is more or less unmaintained. It was a hackweek project from two years ago and only really reached an early prototype stage. We never managed to implement a satisfactory behavior when Jinja2 templating was involved in the salt yaml files (and I doubt there's a straightforward way to pull it off :-/ ), which was a bit of a death sentence. We haven't really touched this project since.

I have not archived the project to keep it open for potential new maintainers, but I personally don't have the time to work on this anymore.

I would be more than happy for any contributions, but as I said, my time to help out is really limited.

[sylvainfaivre]

OK, sorry I won't be able to help, I'm more of a sysadmin than a developer. It might be a good idea to add a paragraph into the README.md with what you replied here, more people would be able to see it.

[dcermak]

sylvainfaivre @.***> writes:

OK, sorry I won't be able to help, I'm more of a sysadmin than a developer. It might be a good idea to add a paragraph into the README.md with what you replied here, more people would be able to see it.

No worries. I've created https://github.com/dcermak/salt-lsp/pull/363 to address your suggestion.

[sylvainfaivre]

great, thanks !

[ekacnet]

I have a branch in my fork: https://github.com/ekacnet/salt-lsp/tree/dev It upgrades pygls to 1.1 and also allows to parse sls files with jinja in it.