search

pappasam / jedi-language-server

A Python language server exclusively for Jedi. If Jedi supports it well, this language server should too.
MIT License
596 stars 45 forks source link

Semantic tokens (fix #137) #231

Open FelipeLema opened 1 year ago

FelipeLema commented 1 year ago

continuation of https://github.com/pappasam/jedi-language-server/pull/196 fixes https://github.com/pappasam/jedi-language-server/issues/137

FelipeLema commented 1 year ago

how can I debug the server? I'm getting an error and no traceback

FelipeLema commented 1 year ago

like, pip install pudb + import pudb; pu.db doesn't work

pappasam commented 1 year ago

Assuming you're still using VSCode, I'd look at VSCode's documentation for how to debug a language server being run by VSCode. For reference, here's coc's docs (the client I use): https://github.com/neoclide/coc.nvim/wiki/Debug-language-server

If you were using coc, you'd set "jedi.trace.server": "verbose", in you coc-settings.json.

pappasam commented 1 year ago

Beyond viewing the logs in your client, you can also run the automated tests locally locally to catch obvious things: https://github.com/pappasam/jedi-language-server#run-tests

image

FelipeLema commented 1 year ago

I meant "how can I debug / get a full stacktrace" when running make test

# make test
poetry run tox -e py
py installed: astroid==2.11.7,attrs==22.1.0,black==22.6.0,cffi==1.15.1,cfgv==3.3.1,click==8.1.3,coverage==6.4.2,dill==0.3.5.1,distlib==0.3.5,docformatter==1.4,docstring-to-markdown==0.10,filelock==3.7.1,greenlet==
0.4.13,hpy==0.0.3,identify==2.5.2,iniconfig==1.1.1,isort==5.10.1,jedi==0.18.1,-e git+ssh://git@github.com/FelipeLema/jedi-language-server.git@f13460f6f4f2022073547155835e6e22017462e8#egg=jedi_language_server&subdi
rectory=../../.worktrees/semantic-tokens-137,lazy-object-proxy==1.7.1,mccabe==0.7.0,mypy==0.971,mypy-extensions==0.4.3,nodeenv==1.7.0,packaging==21.3,parso==0.8.3,pathspec==0.9.0,platformdirs==2.5.2,pluggy==1.0.0,
pre-commit==2.20.0,py==1.11.0,pydantic==1.9.1,pygls==0.12,PyHamcrest==2.0.3,pylint==2.13.9,pyparsing==3.0.9,pytest==7.1.2,pytest-cov==3.0.0,python-jsonrpc-server==0.4.0,PyYAML==6.0,readline==6.2.4.1,six==1.16.0,to
ml==0.10.2,toml-sort==0.20.0,tomli==2.0.1,tomlkit==0.11.1,tox==3.25.1,tox-asdf==0.1.0,typeguard==2.13.3,typing_extensions==4.3.0,ujson==5.4.0,untokenize==0.1.1,virtualenv==20.16.2,wrapt==1.14.1
py run-test-pre: PYTHONHASHSEED='2551634289'
py run-test: commands[0] | poetry install
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: jedi-language-server (0.37.0)
py run-test: commands[1] | poetry run black --check --diff jedi_language_server tests
All done! ✨ 🍰 ✨
55 files would be left unchanged.
py run-test: commands[2] | poetry run docformatter --check --recursive jedi_language_server tests
py run-test: commands[3] | poetry run isort --check jedi_language_server tests
Skipped 1 files
py run-test: commands[4] | poetry run mypy jedi_language_server
Success: no issues found in 9 source files
py run-test: commands[5] | poetry run pylint jedi_language_server tests

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

py run-test: commands[6] | poetry run pytest tests
================================================================================================ test session starts ================================================================================================
platform darwin -- Python 3.9.12[pypy-7.3.10-alpha], pytest-7.1.2, pluggy-1.0.0
cachedir: .tox/py/.pytest_cache
rootdir: /home/xxxx/…/semantic-tokens-137
plugins: typeguard-2.13.3, cov-3.0.0
collected 54 items                                                                                                                                                                                                  

tests/test_cli.py ..                                                                                                                                                                                          [  3%]
tests/lsp_tests/test_completion.py ....                                                                                                                                                                       [ 11%]
tests/lsp_tests/test_definition.py .                                                                                                                                                                          [ 12%]
tests/lsp_tests/test_diagnostics.py ...                                                                                                                                                                       [ 18%]
tests/lsp_tests/test_document_symbol.py ..                                                                                                                                                                    [ 22%]
tests/lsp_tests/test_highlighting.py ..............                                                                                                                                                           [ 48%]
tests/lsp_tests/test_hover.py .....                                                                                                                                                                           [ 57%]
tests/lsp_tests/test_init_options.py .                                                                                                                                                                        [ 59%]
tests/lsp_tests/test_refactoring.py ........                                                                                                                                                                  [ 74%]
tests/lsp_tests/test_references.py ........                                                                                                                                                                   [ 88%]
tests/lsp_tests/test_semantic_tokens.py FFF                                                                                                                                                                   [ 94%]
tests/lsp_tests/test_signature.py ..                                                                                                                                                                          [ 98%]
tests/lsp_tests/test_workspace_symbol.py .                                                                                                                                                                    [100%]

===================================================================================================== FAILURES ======================================================================================================
_________________________________________________________________________________________ test_semantic_tokens_full_import __________________________________________________________________________________________

    def test_semantic_tokens_full_import():
        """Tests tokens for 'import name1 as name2'.

        Test Data: tests/test_data/semantic_tokens/semantic_tokens_test1.py
        """
        with session.LspSession() as ls_session:
            ls_session.initialize()
            uri = as_uri(SEMANTIC_TEST_ROOT / "semantic_tokens_test1.py")
>           actual = ls_session.text_doc_semantic_tokens_full(
                {
                    "textDocument": {"uri": uri},
                }
            )

tests/lsp_tests/test_semantic_tokens.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/lsp_test_client/session.py:188: in text_doc_semantic_tokens_full
    return fut.result()
../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:446: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = None

    def __get_result(self):
        if self._exception:
            try:
>               raise self._exception
E               pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?

../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:391: JsonRpcInvalidParams
----------------------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------------------
ERROR:pygls.protocol:Exception occurred for message "34f45df1-f0d5-44d3-a13e-bba85238ed00": {'code': -32602, 'message': "AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?", 'data': "{'tr
aceback': []}"}
NoneType: None
/home/xxxx/…/pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 2 leaked semaphore objects to clean up at shutdo
wn
  warnings.warn('resource_tracker: There appear to be %d '
_______________________________________________________________________________________ test_semantic_tokens_full_import_from _______________________________________________________________________________________

    def test_semantic_tokens_full_import_from():
        """Tests tokens for 'from name1 import name2 as name3'.

        Test Data: tests/test_data/semantic_tokens/semantic_tokens_test2.py
        """
        with session.LspSession() as ls_session:
            ls_session.initialize()
            uri = as_uri(SEMANTIC_TEST_ROOT / "semantic_tokens_test2.py")
>           actual = ls_session.text_doc_semantic_tokens_full(
                {
                    "textDocument": {"uri": uri},
                }
            )

tests/lsp_tests/test_semantic_tokens.py:47: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/lsp_test_client/session.py:188: in text_doc_semantic_tokens_full
    return fut.result()
../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:446: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = None

    def __get_result(self):
        if self._exception:
            try:
>               raise self._exception
E               pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?

../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:391: JsonRpcInvalidParams
----------------------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------------------
ERROR:pygls.protocol:Exception occurred for message "caf02cc1-5bed-485a-8c71-619a99b32d12": {'code': -32602, 'message': "AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?", 'data': "{'tr
aceback': []}"}
NoneType: None
/home/xxxx/…/pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 2 leaked semaphore objects to clean up at shutdo
wn
  warnings.warn('resource_tracker: There appear to be %d '
______________________________________________________________________________________ test_semantic_tokens_range_import_from _______________________________________________________________________________________

    def test_semantic_tokens_range_import_from():
        """Tests tokens for 'from name1 import name2 as name3'.

        Test Data: tests/test_data/semantic_tokens/semantic_tokens_test2.py
        """
        with session.LspSession() as ls_session:
            ls_session.initialize()
            uri = as_uri(SEMANTIC_TEST_ROOT / "semantic_tokens_test2.py")
>           actual = ls_session.text_doc_semantic_tokens_range(
                {
                    "textDocument": {"uri": uri},
                    "range": {
                        "start": {"line": 1, "character": 1},
                        "end": {"line": 2, "character": 0},
                    },
                }
            )

tests/lsp_tests/test_semantic_tokens.py:76: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/lsp_test_client/session.py:196: in text_doc_semantic_tokens_range
    return fut.result()
../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:446: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = None

    def __get_result(self):
        if self._exception:
            try:
>               raise self._exception
E               pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?

../../../pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/concurrent/futures/_base.py:391: JsonRpcInvalidParams
----------------------------------------------------------------------------------------------- Captured stderr call ------------------------------------------------------------------------------------------------
ERROR:pygls.protocol:Exception occurred for message "0b095d8e-8eb8-43ae-b05c-eb453f8b157e": {'code': -32602, 'message': "AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?", 'data': "{'tr
aceback': []}"}
NoneType: None
/home/xxxx/…/pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/lib/pypy3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 2 leaked semaphore objects to clean up at shutdo
wn
  warnings.warn('resource_tracker: There appear to be %d '
============================================================================================== short test summary info ==============================================================================================
FAILED tests/lsp_tests/test_semantic_tokens.py::test_semantic_tokens_full_import - pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?
FAILED tests/lsp_tests/test_semantic_tokens.py::test_semantic_tokens_full_import_from - pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?
FAILED tests/lsp_tests/test_semantic_tokens.py::test_semantic_tokens_range_import_from - pyls_jsonrpc.exceptions.JsonRpcInvalidParams: AttributeError: 'tuple' object has no attribute 'line'. Did you mean: index?
=========================================================================================== 3 failed, 51 passed in 56.45s ===========================================================================================
ERROR: InvocationError for command /home/xxxx/…/semantic-tokens-137/.pypy_dev_env/bin/poetry run pytest tests (exited with code 1)
______________________________________________________________________________________________________ summary ______________________________________________________________________________________________________
ERROR:   py: commands failed
make: *** [test] Error 1
#
pappasam commented 1 year ago

Not sure, but if you figure it out, do let me know!

quanzhuo commented 1 year ago

I tested this patch, no semantic tokens is retured.

[Trace - 14:19:40] Sending request 'textDocument/semanticTokens/full - (29)'.
Params: {
    "textDocument": {
        "uri": "file:///home/quan/%E4%B8%8B%E8%BD%BD/test-code/Python-Sudoku/util.py"
    }
}

[Trace - 14:19:40] Received notification 'window/logMessage'.
Params: {
    "type": 4,
    "message": "semantic_tokens_full file:///home/quan/%E4%B8%8B%E8%BD%BD/test-code/Python-Sudoku/util.py "
}

semantic_tokens_full file:///home/quan/%E4%B8%8B%E8%BD%BD/test-code/Python-Sudoku/util.py 
[Trace - 14:19:40] Received response 'textDocument/semanticTokens/full - (29)' in 4ms.
Result: {
    "data": []
}
FelipeLema commented 1 year ago

yup, this is not even passing its own unit tests. I'll change this PR from draft to "actually a PR" when people can test it out