LSP->Find References got UnicodeDecodeError error

[sleetdrop]

Describe the bug LSP->Find References doesn't work, and shows error related to UnicodeDecodeError

To Reproduce Steps to reproduce the behavior:

1. Right Click a PHP Class Name
2. Click LSP->Find References
3. Type Ctrl+` to show console
4. See the following Error Traceback (most recent call last): File "/Users/foobar/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/references.py", line 100, in File "/Users/foobar/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/references.py", line 118, in _handle_response File "/Users/foobar/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/references.py", line 156, in _show_references_in_output_panel File "/Users/foobar/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/references.py", line 200, in _group_locations_by_uri File "/Users/foobar/Library/Application Support/Sublime Text 3/Installed Packages/LSP.sublime-package/plugin/core/views.py", line 304, in get_line File "./python3.3/linecache.py", line 15, in getline File "./python3.3/linecache.py", line 41, in getlines File "./python3.3/linecache.py", line 127, in updatecache File "./python3.3/codecs.py", line 313, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 12: invalid continuation byte

Expected behavior Show the class references below.

Environment (please complete the following information):

• OS: macOS 13.2.1
• Sublime Text version: 4147
• LSP version: 1.23.0
• Language servers used: intelephense

Additional context My project have mixed encodings, utf8 and gbk

[rchl]

Can you provide logs from the LSP: Log Panel?

[rwols]

As a workaround I think you can set "show_references_in_quick_panel" to true in LSP's preferences.

[sleetdrop]

Can you provide logs from the LSP: Log Panel?

For privacy concerns, I had changed the filename to {a,b,c...}.php

:: [10:28:04.963] --> LSP-intelephense textDocument/references (5): {'context': {'includeDeclaration': False}, 'workDoneToken': '$ublime-5', 'position': {'line': 9, 'character': 20}, 'textDocument': {'uri': 'file:///path/to/myproject/a.php'}} :: [10:28:05.015] <<< LSP-intelephense (5) (duration: 51ms): [{'range': {'start': {'line': 1434, 'character': 39}, 'end': {'line': 1434, 'character': 58}}, 'uri': 'file:///path/to/myproject/b.php'}, {'range': {'start': {'line': 1439, 'character': 39}, 'end': {'line': 1439, 'character': 58}}, 'uri': 'file:///path/to/myproject/b.php'}, {'range': {'start': {'line': 1442, 'character': 52}, 'end': {'line': 1442, 'character': 71}}, 'uri': 'file:///path/to/myproject/b.php'}, {'range': {'start': {'line': 604, 'character': 35}, 'end': {'line': 604, 'character': 54}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 607, 'character': 48}, 'end': {'line': 607, 'character': 67}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 631, 'character': 31}, 'end': {'line': 631, 'character': 50}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 1210, 'character': 27}, 'end': {'line': 1210, 'character': 46}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 1288, 'character': 23}, 'end': {'line': 1288, 'character': 42}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 1994, 'character': 19}, 'end': {'line': 1994, 'character': 38}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 2085, 'character': 17}, 'end': {'line': 2085, 'character': 36}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 2243, 'character': 27}, 'end': {'line': 2243, 'character': 46}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 2323, 'character': 14}, 'end': {'line': 2323, 'character': 33}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 3405, 'character': 23}, 'end': {'line': 3405, 'character': 42}}, 'uri': 'file:///path/to/myproject/c.php'}, {'range': {'start': {'line': 5, 'character': 22}, 'end': {'line': 5, 'character': 41}}, 'uri': 'file:///path/to/myproject/d.php'}, {'range': {'start': {'line': 7, 'character': 2}, 'end': {'line': 7, 'character': 21}}, 'uri': 'file:///path/to/myproject/d.php'}, {'range': {'start': {'line': 8, 'character': 8}, 'end': {'line': 8, 'character': 27}}, 'uri': 'file:///path/to/myproject/d.php'}, {'range': {'start': {'line': 95, 'character': 27}, 'end': {'line': 95, 'character': 46}}, 'uri': 'file:///path/to/myproject/e.php'}, {'range': {'start': {'line': 930, 'character': 35}, 'end': {'line': 930, 'character': 54}}, 'uri': 'file:///path/to/myproject/f.php'}, {'range': {'start': {'line': 118, 'character': 27}, 'end': {'line': 118, 'character': 46}}, 'uri': 'file:///path/to/myproject/g.php'}, {'range': {'start': {'line': 53, 'character': 29}, 'end': {'line': 53, 'character': 48}}, 'uri': 'file:///path/to/myproject/h.php'}, {'range': {'start': {'line': 90, 'character': 18}, 'end': {'line': 90, 'character': 37}}, 'uri': 'file:///path/to/myproject/i.php'}, {'range': {'start': {'line': 582, 'character': 31}, 'end': {'line': 582, 'character': 50}}, 'uri': 'file:///path/to/myproject/j.php'}, {'range': {'start': {'line': 182, 'character': 39}, 'end': {'line': 182, 'character': 58}}, 'uri': 'file:///path/to/myproject/k.php'}]

[sleetdrop]

As a workaround I think you can set "show_references_in_quick_panel" to true in LSP's preferences.

Thanks, this workaround method just works for me.

[rchl]

Were there any non-ascii / non-utf-8 characters in that output? I suppose the issue was with file paths including some "tricky" characters. If so, can you paste those characters here?

[sleetdrop]

Were there any non-ascii / non-utf-8 characters in that output? I suppose the issue was with file paths including some "tricky" characters. If so, can you paste those characters here?

There is no non-ascii / non-utf-8 characters, the file path which I replaced is all ascii characters and without spaces either.

I do some digging in the python docs https://docs.python.org/3/library/linecache.html

It says linecache is mainly for python source code, it use tokenize.detect_encoding() to detect encoding. In tokenize.detect_encoding() doc has this line. It detects the encoding from the presence of a UTF-8 BOM or an encoding cookie as specified in PEP 263.

Maybe I should add # -*- coding: gbk -*- at the head of my php source file to let linecache to detect the encoding correctly.

If so, we should use another way to implement this feature instead of linecache