search

puremourning / vimspector

vimspector - A multi-language debugging system for Vim
http://puremourning.github.io/vimspector-web
Apache License 2.0
4.11k stars 175 forks source link

UnicodeEncodeError for the character "ō" #671

Closed waivek closed 2 years ago

waivek commented 2 years ago

Description

When launching vimspector or using the balloon eval feature, the character "ō" causes a UnicodeEncodeError in python.

vimspector_encode v01

Works in vim?

Reproducable in Vim

Works with clean config?

Yes

Sample project works?

Reproducable with sample project

Minimal Reproduction

Open a python file with the contents:

japanese_name = "Yoshifumi Kondō"
raise Exception

Launch vimspector using: :call vimspector#LaunchWithSettings({ 'configuration': 'run' })

Expected Behaviour

Expected no error messages during startup / balloon eval.

Actual Behaviour

A vim error trace is produced with the following output

Error detected while processing function <SNR>156__OnServerData:
line    6:
--- Logging error ---
Traceback (most recent call last):
  File "C:\Program Files\Python310\Lib\logging\__init__.py", line 1101, in emit
    stream.write(msg + self.terminator)
  File "C:\Program Files\Python310\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u014d' in position 336: character maps to <undefined>
Call stack:
  File "<string>", line 1, in <module>
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_session.py", line 463, in OnChannelData
    self._connection.OnData( data )
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_adapter_connection.py", line 179, in OnData
    self._ReadBody()
  File "C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\debug_adapter_connection.py", line 257, in _ReadBody
    self._logger.debug( 'Message received: {0}'.format( message ) )
Message: 'Message received: {\'seq\': 24, \'type\': \'response\', \'request_seq\': 15, \'success\': True, \'command\': \'variables\', \'body\': {\'variables\': [{\'name\': \'special variables\', \'value\': \'\', \'type\': \'\', \'evaluateName\': \'special variables\', \'variablesReference\': 7}, {\'name\': \'japanese_name\', \'value\': "\'Yoshifumi Kondō\'", \'type\': \'str\', \'evaluateName\': \'japanese_name\', \'variablesReference\': 0, \'presentationHint\': {\'attributes\': [\'rawString\']}}]}}'
Arguments: ()

Additional information

No response

Installation Type

Plugin Manager (git clone)

Vimspector version

3378018bc1cdd1d9b70734c3e970bc52fd983415

Debug Info

Vimspector Debug Info
--------------------------------------------------------------------------------
ConnectionType: job
Adapter: 
--------------------------------------------------------------------------------
{
  "command": [
    "C:\\Program Files\\Python310\\python.exe",
    "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"
  ],
  "configuration": {
    "python": "C:\\Program Files\\Python310\\python.exe",
    "subProcess": false
  },
  "name": "debugpy",
  "env": {},
  "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python"
}
--------------------------------------------------------------------------------
Configuration: 
--------------------------------------------------------------------------------
{
  "adapter": "debugpy",
  "configuration": {
    "request": "launch",
    "type": "python",
    "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python",
    "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py",
    "stopOnEntry": false,
    "console": "integratedTerminal",
    "justMyCode": false,
    "purpose": [
      "debug-in-terminal"
    ]
  },
  "breakpoints": {
    "exception": {
      "raised": "N",
      "uncaught": "",
      "userUnhandled": ""
    }
  }
}
--------------------------------------------------------------------------------
API Prefix: 
Launch/Init: True / True
Workspace Root: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python
Launch Config: 
--------------------------------------------------------------------------------
{
  "python": "C:\\Program Files\\Python310\\python.exe",
  "subProcess": false,
  "request": "launch",
  "type": "python",
  "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python",
  "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py",
  "stopOnEntry": false,
  "console": "integratedTerminal",
  "justMyCode": false,
  "purpose": [
    "debug-in-terminal"
  ],
  "name": "test"
}
--------------------------------------------------------------------------------
Server Capabilities: 
--------------------------------------------------------------------------------
{
  "supportsCompletionsRequest": true,
  "supportsConditionalBreakpoints": true,
  "supportsConfigurationDoneRequest": true,
  "supportsDebuggerProperties": true,
  "supportsDelayedStackTraceLoading": true,
  "supportsEvaluateForHovers": true,
  "supportsExceptionInfoRequest": true,
  "supportsExceptionOptions": true,
  "supportsFunctionBreakpoints": true,
  "supportsHitConditionalBreakpoints": true,
  "supportsLogPoints": true,
  "supportsModulesRequest": true,
  "supportsSetExpression": true,
  "supportsSetVariable": true,
  "supportsValueFormattingOptions": true,
  "supportsTerminateDebuggee": true,
  "supportsGotoTargetsRequest": true,
  "supportsClipboardContext": true,
  "exceptionBreakpointFilters": [
    {
      "filter": "raised",
      "label": "Raised Exceptions",
      "default": false,
      "description": "Break whenever any exception is raised."
    },
    {
      "filter": "uncaught",
      "label": "Uncaught Exceptions",
      "default": true,
      "description": "Break when the process is exiting due to unhandled exception."
    },
    {
      "filter": "userUnhandled",
      "label": "User Uncaught Exceptions",
      "default": false,
      "description": "Break when exception escapes into library code."
    }
  ],
  "supportsStepInTargetsRequest": true
}
--------------------------------------------------------------------------------
Line Breakpoints: 
--------------------------------------------------------------------------------
{
  "C:\\Users\\vivek\\Documents\\samples\\letterboxd_ten.py": [],
  "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\dbg.py": [],
  "C:\\Users\\vivek\\vimfiles\\ide.vim": [],
  "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py": []
}
--------------------------------------------------------------------------------
Func Breakpoints: 
--------------------------------------------------------------------------------
[]
--------------------------------------------------------------------------------
Ex Breakpoints: 
--------------------------------------------------------------------------------
{
  "filters": [
    "uncaught"
  ],
  "exceptionOptions": []
}
--------------------------------------------------------------------------------

Log file

2022-09-19 20:56:54,273 - INFO - **** INITIALISING NEW VIMSPECTOR SESSION ****
2022-09-19 20:56:54,274 - INFO - API is: 
2022-09-19 20:56:54,274 - INFO - VIMSPECTOR_HOME = C:\Users\vivek\vimfiles\plugged\vimspector\python3\vimspector\..\..
2022-09-19 20:56:54,404 - INFO - gadgetDir = C:\Users\vivek\vimfiles\plugged\vimspector\gadgets\windows
2022-09-19 20:56:54,405 - INFO - User requested start debug session with {}
2022-09-19 20:56:54,408 - DEBUG - Reading configurations from: None
2022-09-19 20:56:54,409 - DEBUG - Reading configurations from: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python\.vimspector.json
2022-09-19 20:56:54,773 - DEBUG - Reading gadget config: C:\Users\vivek\vimfiles\plugged\vimspector\gadgets\windows\.gadgets.json
2022-09-19 20:56:54,776 - DEBUG - Reading gadget config: None
2022-09-19 20:56:56,479 - INFO - Configuration: {"adapter": "debugpy", "configuration": {"request": "launch", "type": "python", "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python", "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py", "stopOnEntry": false, "console": "integratedTerminal", "justMyCode": false, "purpose": ["debug-in-terminal"]}, "breakpoints": {"exception": {"raised": "N", "uncaught": "", "userUnhandled": ""}}}
2022-09-19 20:56:56,479 - INFO - Adapter: {"command": ["C:\\Program Files\\Python310\\python.exe", "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"], "configuration": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false}, "name": "debugpy"}
2022-09-19 20:56:56,857 - DEBUG - min_width/height: 149/50, actual: 172/38 - result: horizontal
2022-09-19 20:56:57,408 - DEBUG - LAUNCH!
2022-09-19 20:56:57,417 - INFO - Starting debug adapter with: {"command": ["C:\\Program Files\\Python310\\python.exe", "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows/debugpy/build/lib/debugpy/adapter"], "configuration": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false}, "name": "debugpy"}
2022-09-19 20:56:57,417 - DEBUG - Connection Type: job
2022-09-19 20:56:57,546 - INFO - Debug Adapter Started
2022-09-19 20:56:57,547 - DEBUG - Sending Message: {"command": "initialize", "arguments": {"adapterID": "debugpy", "clientID": "vimspector", "clientName": "vimspector", "linesStartAt1": true, "columnsStartAt1": true, "locale": "en_GB", "pathFormat": "path", "supportsVariableType": true, "supportsVariablePaging": false, "supportsRunInTerminalRequest": true, "supportsMemoryReferences": true}, "seq": 0, "type": "request"}
2022-09-19 20:56:58,243 - DEBUG - Message received: {'seq': 1, 'type': 'event', 'event': 'output', 'body': {'category': 'telemetry', 'output': 'ptvsd', 'data': {'packageVersion': '1.6.3'}}}
2022-09-19 20:56:58,244 - DEBUG - Message received: {'seq': 2, 'type': 'event', 'event': 'output', 'body': {'category': 'telemetry', 'output': 'debugpy', 'data': {'packageVersion': '1.6.3'}}}
2022-09-19 20:56:58,244 - DEBUG - Message received: {'seq': 3, 'type': 'response', 'request_seq': 0, 'success': True, 'command': 'initialize', 'body': {'supportsCompletionsRequest': True, 'supportsConditionalBreakpoints': True, 'supportsConfigurationDoneRequest': True, 'supportsDebuggerProperties': True, 'supportsDelayedStackTraceLoading': True, 'supportsEvaluateForHovers': True, 'supportsExceptionInfoRequest': True, 'supportsExceptionOptions': True, 'supportsFunctionBreakpoints': True, 'supportsHitConditionalBreakpoints': True, 'supportsLogPoints': True, 'supportsModulesRequest': True, 'supportsSetExpression': True, 'supportsSetVariable': True, 'supportsValueFormattingOptions': True, 'supportsTerminateDebuggee': True, 'supportsGotoTargetsRequest': True, 'supportsClipboardContext': True, 'exceptionBreakpointFilters': [{'filter': 'raised', 'label': 'Raised Exceptions', 'default': False, 'description': 'Break whenever any exception is raised.'}, {'filter': 'uncaught', 'label': 'Uncaught Exceptions', 'default': True, 'description': 'Break when the process is exiting due to unhandled exception.'}, {'filter': 'userUnhandled', 'label': 'User Uncaught Exceptions', 'default': False, 'description': 'Break when exception escapes into library code.'}], 'supportsStepInTargetsRequest': True}}
2022-09-19 20:56:58,246 - DEBUG - Sending Message: {"command": "launch", "arguments": {"python": "C:\\Program Files\\Python310\\python.exe", "subProcess": false, "request": "launch", "type": "python", "cwd": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python", "program": "C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py", "stopOnEntry": false, "console": "integratedTerminal", "justMyCode": false, "purpose": ["debug-in-terminal"], "name": "test"}, "seq": 1, "type": "request"}
2022-09-19 20:56:58,257 - DEBUG - Message received: {'seq': 4, 'type': 'request', 'command': 'runInTerminal', 'arguments': {'kind': 'integrated', 'title': 'Python Debug Console', 'args': ['C:\\Program Files\\Python310\\python.exe', 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\gadgets\\windows\\debugpy\\build\\lib\\debugpy\\adapter/../..\\debugpy\\launcher', '2607', '--', 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py'], 'env': {}, 'cwd': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python'}}
2022-09-19 20:56:58,622 - DEBUG - Sending Message: {"seq": 2, "type": "response", "request_seq": 4, "command": "runInTerminal", "body": {"processId": 10532}, "success": true}
2022-09-19 20:57:01,291 - DEBUG - Message received: {'seq': 5, 'type': 'event', 'event': 'initialized'}
2022-09-19 20:57:01,291 - DEBUG - Sending Message: {"command": "setFunctionBreakpoints", "arguments": {"breakpoints": []}, "seq": 3, "type": "request"}
2022-09-19 20:57:01,292 - DEBUG - Sending Message: {"command": "setExceptionBreakpoints", "arguments": {"filters": ["uncaught"], "exceptionOptions": []}, "seq": 4, "type": "request"}
2022-09-19 20:57:01,302 - DEBUG - Message received: {'seq': 6, 'type': 'response', 'request_seq': 3, 'success': True, 'command': 'setFunctionBreakpoints', 'body': {'breakpoints': []}}
2022-09-19 20:57:01,303 - DEBUG - Message received: {'seq': 7, 'type': 'response', 'request_seq': 4, 'success': True, 'command': 'setExceptionBreakpoints'}
2022-09-19 20:57:01,303 - DEBUG - Sending Message: {"command": "configurationDone", "seq": 5, "type": "request"}
2022-09-19 20:57:01,314 - DEBUG - Message received: {'seq': 8, 'type': 'response', 'request_seq': 5, 'success': True, 'command': 'configurationDone'}
2022-09-19 20:57:01,315 - DEBUG - Message received: {'seq': 9, 'type': 'response', 'request_seq': 1, 'success': True, 'command': 'launch'}
2022-09-19 20:57:01,316 - DEBUG - Sending Message: {"command": "threads", "seq": 6, "type": "request"}
2022-09-19 20:57:01,316 - DEBUG - Message received: {'seq': 10, 'type': 'event', 'event': 'process', 'body': {'startMethod': 'launch', 'isLocalProcess': True, 'systemProcessId': 10928, 'name': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py', 'pointerSize': 64}}
2022-09-19 20:57:01,317 - INFO - User Msg: The debuggee was started: C:\Users\vivek\vimfiles\plugged\vimspector\support\test\python\simple_python\encode_error.py
2022-09-19 20:57:01,325 - DEBUG - Message received: {'seq': 11, 'type': 'event', 'event': 'thread', 'body': {'reason': 'started', 'threadId': 1}}
2022-09-19 20:57:01,327 - DEBUG - Message received: {'seq': 12, 'type': 'response', 'request_seq': 6, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,328 - DEBUG - Sending Message: {"command": "threads", "seq": 7, "type": "request"}
2022-09-19 20:57:01,353 - DEBUG - Message received: {'seq': 13, 'type': 'response', 'request_seq': 7, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,431 - DEBUG - Message received: {'seq': 14, 'type': 'event', 'event': 'stopped', 'body': {'reason': 'exception', 'description': 'exception: no description', 'threadId': 1, 'preserveFocusHint': False, 'text': 'Exception       (note: full exception trace is shown but execution is paused at: _run_module_as_main)', 'allThreadsStopped': True}}
2022-09-19 20:57:01,431 - WARNING - User Msg: Paused in thread 1 due to exception: no description(exception): Exception       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
2022-09-19 20:57:01,559 - DEBUG - Sending Message: {"command": "threads", "seq": 8, "type": "request"}
2022-09-19 20:57:01,564 - DEBUG - Message received: {'seq': 15, 'type': 'response', 'request_seq': 8, 'success': True, 'command': 'threads', 'body': {'threads': [{'id': 1, 'name': 'MainThread'}]}}
2022-09-19 20:57:01,564 - DEBUG - Sending Message: {"command": "stackTrace", "arguments": {"threadId": 1}, "seq": 9, "type": "request"}
2022-09-19 20:57:01,588 - DEBUG - Message received: {'seq': 16, 'type': 'response', 'request_seq': 9, 'success': True, 'command': 'stackTrace', 'body': {'stackFrames': [{'id': 2, 'name': '<module>', 'line': 2, 'column': 1, 'source': {'path': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py', 'sourceReference': 0}}, {'id': 3, 'name': '_run_code', 'line': 86, 'column': 1, 'source': {'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py', 'sourceReference': 0}, 'presentationHint': 'subtle'}, {'id': 4, 'name': '_run_module_as_main (Current frame)', 'line': 196, 'column': 1, 'source': {'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py', 'sourceReference': 0}, 'presentationHint': 'subtle'}], 'totalFrames': 3}}
2022-09-19 20:57:01,611 - DEBUG - Sending Message: {"command": "scopes", "arguments": {"frameId": 2}, "seq": 10, "type": "request"}
2022-09-19 20:57:01,614 - DEBUG - Message received: {'seq': 17, 'type': 'event', 'event': 'module', 'body': {'reason': 'new', 'module': {'id': 0, 'name': '__main__', 'path': 'C:\\Users\\vivek\\vimfiles\\plugged\\vimspector\\support\\test\\python\\simple_python\\encode_error.py'}}}
2022-09-19 20:57:01,614 - DEBUG - Message received: {'seq': 18, 'type': 'event', 'event': 'module', 'body': {'reason': 'new', 'module': {'id': 1, 'name': 'runpy', 'path': 'C:\\Program Files\\Python310\\Lib\\runpy.py'}}}
2022-09-19 20:57:01,673 - DEBUG - Message received: {'seq': 19, 'type': 'response', 'request_seq': 10, 'success': True, 'command': 'scopes', 'body': {'scopes': [{'name': 'Locals', 'variablesReference': 5, 'expensive': False, 'presentationHint': 'locals', 'source': {}}, {'name': 'Globals', 'variablesReference': 6, 'expensive': False, 'source': {}}]}}
2022-09-19 20:57:01,674 - DEBUG - Sending Message: {"command": "variables", "arguments": {"variablesReference": 5}, "seq": 11, "type": "request"}
2022-09-19 20:57:02,285 - DEBUG - Message received: {'seq': 21, 'type': 'event', 'event': 'output', 'body': {'output': 'pydevd warning: Computing repr of special variables () was slow (took 0.42s)\n', 'category': 'important', 'source': {}}}

Vim version

VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Sep 11 2022 22:03:01)
MS-Windows 64-bit GUI version with OLE support
Included patches: 1-447
Compiled by appveyor@APPVYR-WIN
Huge version with GUI.  Features included (+) or not (-):
+acl                +ex_extra           +mzscheme/dyn       -tcl
+arabic             +extra_search       +netbeans_intg      -termguicolors
+autocmd            -farsi              +num64              +terminal
+autochdir          +file_in_path       +ole                -termresponse
+autoservername     +find_in_path       +packages           +textobjects
+balloon_eval       +float              +path_extra         +textprop
-balloon_eval_term  +folding            +perl/dyn           -tgetent
+browse             -footer             +persistent_undo    +timers
++builtin_terms     +gettext/dyn        +popupwin           +title
+byte_offset        -hangul_input       -postscript         +toolbar
+channel            +iconv/dyn          +printer            +user_commands
+cindent            +insert_expand      +profile            +vartabs
+clientserver       +ipv6               +python/dyn         +vertsplit
+clipboard          +job                +python3/dyn        +vim9script
+cmdline_compl      +jumplist           +quickfix           +viminfo
+cmdline_hist       +keymap             +reltime            +virtualedit
+cmdline_info       +lambda             +rightleft          +visual
+comments           +langmap            +ruby/dyn           +visualextra
+conceal            +libcall            +scrollbind         +vreplace
+cryptv             +linebreak          +signs              -vtp
+cscope             +lispindent         +smartindent        +wildignore
+cursorbind         +listcmds           +sodium/dyn         +wildmenu
+cursorshape        +localmap           +sound              +windows
+dialog_con_gui     +lua/dyn            +spell              +writebackup
+diff               +menu               +startuptime        -xfontset
+digraphs           +mksession          +statusline         -xim
+directx            +modify_fname       -sun_workshop       +xpm_w32
-dnd                +mouse              +syntax             -xterm_save
-ebcdic             +mouseshape         +tag_binary         
+emacs_tags         +multi_byte_ime/dyn -tag_old_static     
+eval               +multi_lang         -tag_any_white      
   system vimrc file: "$VIM\vimrc"
     user vimrc file: "$HOME\_vimrc"
 2nd user vimrc file: "$HOME\vimfiles\vimrc"
 3rd user vimrc file: "$VIM\_vimrc"
      user exrc file: "$HOME\_exrc"
  2nd user exrc file: "$VIM\_exrc"
  system gvimrc file: "$VIM\gvimrc"
    user gvimrc file: "$HOME\_gvimrc"
2nd user gvimrc file: "$HOME\vimfiles\gvimrc"
3rd user gvimrc file: "$VIM\_gvimrc"
       defaults file: "$VIMRUNTIME\defaults.vim"
    system menu file: "$VIMRUNTIME\menu.vim"
Compilation: cl -c /W3 /GF /nologo -I. -Iproto -DHAVE_PATHDEF -DWIN32  -DFEAT_CSCOPE -DFEAT_TERMINAL 
-DFEAT_SOUND -DFEAT_NETBEANS_INTG -DFEAT_JOB_CHANNEL -DFEAT_IPV6   -DFEAT_XPM_W32 
-DHAVE_SODIUM -DDYNAMIC_SODIUM -DDYNAMIC_SODIUM_DLL=\"libsodium.dll\" /I "C:\libsodium\include"

Python version

No response

Neovim diagnostics

No response

Operating System

OS: Windows 10 20H2

Declaration

puremourning commented 2 years ago

Thanks. This is covered by #96