Open jaraco opened 4 years ago
My first idea is that this is possibly a broken key binding that moves the cursor beyond the text length. If we have a reliable way to reproduce it, that would be great.
This traceback here is what happens during the rendering of the output to the screen, the actual error is a buffer manipulation some time before this.
I usually notice the issue when I've been typing into the shell while it's starting up. I suspect, though can't confirm, that I'm both entering and deleting characters. I tried replicating the issue by entering and deleting lots of characters during startup, but thusfar has been unsuccessful in triggering the behavior. I also haven't experienced the issue since, so maybe 3.0.5 addresses the issue. I'll do my best to capture more precisely the conditions that cause the issue next time it happens.
This issue pops up quite often on my end when opening alacritty with xonsh/xonsh#3522 as default shell.
The issue just happened again. I started up a new shell and was typing into the shell as it was starting up:
~ $ cd ~/p/pypa/setuptools
Unhandled exception in event loop:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/eventloop/utils.py", line 72, in schedule
func()
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 456, in redraw
self._redraw()
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 523, in _redraw
self.context.run(run_in_context)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 509, in run_in_context
self.renderer.render(self, self.layout)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/renderer.py", line 617, in render
layout.container.preferred_height(size.columns, size.rows).preferred,
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 792, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 2595, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1620, in preferred_height
return self._merge_dimensions(
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1648, in _merge_dimensions
preferred = get_preferred()
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1613, in preferred_content_height
return self.content.preferred_height(
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 639, in preferred_height
content = self.create_content(width, height=1) # Pass a dummy '1' as height.
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 761, in create_content
document = buffer.document
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 524, in document
return self._document_cache[
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/cache.py", line 98, in __missing__
result = self.get_value(*key)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/document.py", line 98, in __init__
assert cursor_position is None or cursor_position <= len(text), AssertionError(
Exception cursor_position=8, len_text=7
Press ENTER to continue...
...
Hitting ENTER emits the exact same error.
I'd say this happens about 1/100 times. I'm unable to replicate it reliably.
As you can see, I'd entered 22 characters plus carriage return (maybe, I'm not certain about the CR), but the buffer manipulation seems to have gone awry before that. I can't remember if I hit backspace. For this command, I give a <20% chance I hit backspace while typing it.
Today it happened again. Again, I'd started up a clean terminal and started typing ls sudoers^h^h^h^h^h^h^h/etc/s
(or thereabouts, when the error occurred):
Unhandled exception in event loop:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 653, in read_from_input
self.key_processor.process_keys()
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 274, in process_keys
self._process_coroutine.send(key_press)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 186, in _process
self._call_handler(matches[-1], key_sequence=buffer[:])
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 329, in _call_handler
handler.call(event)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_bindings.py", line 101, in call
self.handler(event)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/bindings/named_commands.py", line 270, in self_insert
event.current_buffer.insert_text(event.data * event.arg)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 1209, in insert_text
self.document = Document(text, cpos)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/document.py", line 98, in __init__
assert cursor_position is None or cursor_position <= len(text), AssertionError(
Exception cursor_position=7, len_text=5
Press ENTER to continue...
~ $ ls /etc/ssh
This time, as you can see, hitting ENTER dismissed the error and I could see my partially-entered command with some autocomplete present:
Here, neither the entered text len('ls /et') == 6
nor the cursor position (6 if the empty prompt is position 0) appear to correspond to the values in the exception.
If you can think of a way that I could add some debugging to prompt_toolkit or xonsh to help capture the issue, I'm happy to do that.
I can reproduce it with approx. every 3rd try by launching a terminal and quickly search for a command in the history (CTRL + R
). The weird thing is that I now get 3 different exceptions here:
cd .local/lib/python3.8/site-packages/compose/
Exception in thread Thread-11:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
Exception in threading.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 1202, in invoke_excepthook
hook(args)
File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 17609, in flush
self.std.flush()
AttributeError: 'NoneType' object has no attribute 'flush'
or
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 24210, in main
return main_xonsh(args)
File "/usr/lib/python3.8/site-packages/xonsh/__amalgam__.py", line 24239, in main_xonsh
shell.shell.cmdloop()
File "/usr/lib/python3.8/site-packages/xonsh/ptk_shell/shell.py", line 197, in cmdloop
line = self.singleline(auto_suggest=auto_suggest)
File "/usr/lib/python3.8/site-packages/xonsh/ptk_shell/shell.py", line 166, in singleline
line = self.prompter.prompt(**prompt_args)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/shortcuts/prompt.py", line 994, in prompt
return self.app.run(set_exception_handler=set_exception_handler)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 811, in run
return loop.run_until_complete(
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 778, in run_async
return await _run_async2()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 760, in _run_async2
result = await _run_async()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 686, in _run_async
self._redraw()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 523, in _redraw
self.context.run(run_in_context)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 509, in run_in_context
self.renderer.render(self, self.layout)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/renderer.py", line 568, in render
layout.container.preferred_height(size.columns, size.rows).preferred,
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 792, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 2595, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1620, in preferred_height
return self._merge_dimensions(
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1648, in _merge_dimensions
preferred = get_preferred()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1613, in preferred_content_height
return self.content.preferred_height(
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 639, in preferred_height
content = self.create_content(width, height=1) # Pass a dummy '1' as height.
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 761, in create_content
document = buffer.document
File "/usr/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 525, in document
self.text, self.cursor_position, self.selection_state
File "/usr/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 416, in text
return self._working_lines[self.working_index]
IndexError: list index out of range
Xonsh encountered an issue during launch
Failback to /bin/sh
or
Unhandled exception in event loop:
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 837, in in_term
await _do_wait_for_enter("Press ENTER to continue...")
File "/usr/lib/python3.8/contextlib.py", line 178, in __aexit__
await self.gen.__anext__()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/run_in_terminal.py", line 114, in in_terminal
app._redraw()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 523, in _redraw
self.context.run(run_in_context)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 509, in run_in_context
self.renderer.render(self, self.layout)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/renderer.py", line 568, in render
layout.container.preferred_height(size.columns, size.rows).preferred,
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 792, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 324, in preferred_height
dimensions = [
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 325, in <listcomp>
c.preferred_height(width, max_available_height) for c in self._all_children
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 2595, in preferred_height
return self.content.preferred_height(width, max_available_height)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1620, in preferred_height
return self._merge_dimensions(
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1648, in _merge_dimensions
preferred = get_preferred()
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/containers.py", line 1613, in preferred_content_height
return self.content.preferred_height(
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 639, in preferred_height
content = self.create_content(width, height=1) # Pass a dummy '1' as height.
File "/usr/lib/python3.8/site-packages/prompt_toolkit/layout/controls.py", line 761, in create_content
document = buffer.document
File "/usr/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 524, in document
return self._document_cache[
File "/usr/lib/python3.8/site-packages/prompt_toolkit/cache.py", line 98, in __missing__
result = self.get_value(*key)
File "/usr/lib/python3.8/site-packages/prompt_toolkit/document.py", line 98, in __init__
assert cursor_position is None or cursor_position <= len(text), AssertionError(
Exception cursor_position=48, len_text=3
Press ENTER to continue...
all triggered by the same repro steps (e.g. compose
which reveals a long path)!?
Today the issue happened again, this time I definitely did not hit backspace when typing the characters:
Unhandled exception in event loop:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 653, in read_from_input
self.key_processor.process_keys()
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 274, in process_keys
self._process_coroutine.send(key_press)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 186, in _process
self._call_handler(matches[-1], key_sequence=buffer[:])
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_processor.py", line 329, in _call_handler
handler.call(event)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/key_bindings.py", line 101, in call
self.handler(event)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/key_binding/bindings/named_commands.py", line 270, in self_insert
event.current_buffer.insert_text(event.data * event.arg)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 1209, in insert_text
self.document = Document(text, cpos)
File "/Users/jaraco/.local/pipx/venvs/xonsh/lib/python3.8/site-packages/prompt_toolkit/document.py", line 98, in __init__
assert cursor_position is None or cursor_position <= len(text), AssertionError(
Exception cursor_position=19, len_text=7
Press ENTER to continue...
~ $ cd ~/p/pypa/setuptools
Cursor was after the t
in setuptools (position 19 if 1 based, 18 if zero-based).
@jaraco I think I found the reason for my problems: fairly large history files (~/.local/share/xonsh/*json
). The observations stated in #1158 and the related xonsh issue #3586 resemble exactly what I experience. I am curious whether this is the case for you, as well.
Today, I noticed something. I started up my shell, and before the prompt was presented, I typed a few characters, but then waited for the prompt to appear, which takes a few seconds. After the prompt appeared, I started typing the rest of the command, after which the error appeared. I've noticed the issue is almost always (if not always) at the beginning of a session. These new findings are consistent with the findings mentioned above, so seem to be a likely cause.
My history directory has 4.5k json files totallying about 45MB:
~ $ len(list(pathlib.Path('~/.local/share/xonsh').expanduser().glob('*.json')))
4458
It's too many files to pass to ls
;)
So I think it's a likely trigger.
Sounds like #1170 is a likely fix.
I've installed that PR in my main environment and will report back if that doesn't work around the issue.
~ $ ~/.local/pipx/venvs/xonsh/bin/python -m pip install git+https://github.com/bobhy/python-prompt-toolkit@th-threadsafe-load-2
I use xonsh as my shell with prompt-toolkit 3.0.4. Occasionally, I'll be typing in the prompt and get this traceback:
Sometimes, I hit enter and the error repeats. Other times, I hit Enter and can continue.
I've just updated to prompt_toolkit==3.0.5. I'll report back here if the issue repeats.