Typeahead Issues in IPython

[mlucool]

Hi,

We'd like to report a few issues with typeahead and IPython. Feel free to point this to the right project itself (prompt-toolkit? IPython?).

The period after the tab causes the tab to be ignored:

time.sleep(8)<ENTER>IPython.cor<TAB>.

The parenthesis after the tab does something weird, autocompleting empty string:

time.sleep(8)<ENTER>np.arang<TAB>(

The ENTER after the tab causes the tab to be ignored:

time.sleep(8)<ENTER>IPython.cor<TAB><ENTER>.

Everything after the xx line is ignored:

time.sleep(8)<ENTER>xx<ENTER>yy<ENTER>.  

I can make gifs of the others, but I thought this showed it clearly:

cc @Carreau

[Carreau]

Let me see if I understand things correctly.

in example 1) while sleep(8) is running, you are typing <ENTER>IPython.cor<TAB>., and you expect to actually get IPython.core. in your buffer, (e was inserted).

Right?

I believe this is because the <TAB> binding is only available when waiting for input, I'm not sure we can do much about this one. In the case IPython.cor<tab> there is a single completion, but if there was many we need to figure out what to do. I'm not even sure we get the right hooks with prompt toolkit in this specific case, but I'll see what we can do.

The "empty string" is kind of expected, you opened a parenthesis and so anything can be a first argument. I know that about never what user wants, and we need to have a better completer in IPython, but I think this is unrelated to this question).

[mlucool]

Right?

Exactly

In the case IPython.cor there is a single completion, I think it working in the case of a single completion is good, but even better is just choosing the first in the list.

The "empty string" is kind of expected

I think the idea was the tab should have been consumed before the ( so no autocomplete is shown

[jonathanslenders]

I think the problem is that prompt_toolkit runs the completions in a background task and it goes on processing the input before the completions are actually computed. I'm not sure there is much we can do about it now, but I'll have another look.

The completion code is an async coroutine, while key bindings are not coroutines. So, we can't use an "await" in any code triggered from within a key binding. (Maybe we can make that possible, and accept coroutines for key bindings, and don't process a new key binding before the previous one is finished.) Quite a bit of effort to make this work, but it would be nice to have, I guess.

[Carreau]

Is there a way to ask prompt toolkit outside of prompt() to not catch the tab key and leave it in the text stream, then get the full text stream back ? Like a raw "This was typed since last interaction", and let IPython process it with a different logic ?

[asmeurer]

I think this is related to https://github.com/prompt-toolkit/python-prompt-toolkit/issues/497

[asmeurer]

One may even consider this to be the same issue as #497, I'm not sure. One problem I had thinking about that issue is that in some instances, like for typeahead, you really want the completer to be blocking/synchronous, whereas in other instances, you really don't. For example, if a completion takes a long time (and since completions can come from any Python function, they can take an arbitrary amount of time), you want to be able to cancel them. If you've ever tried to tab complete an enormous directory in bash you've experienced this. In practice, the Jedi completions from IPython can be slower, depending on the code they are completing. So you end up going back and forth between "I want completions to block" and "I don't want completions to block".

My idea in https://github.com/prompt-toolkit/python-prompt-toolkit/issues/497#issuecomment-320371107 was to block for some short amount of time, say 0.5 seconds (but we should experiment to find a good value), during which any typed text would act like typeahead after the completion. Then, if the completer takes longer than that, act non-blocking like it does now and have typed text cancel the completion. I wonder if there are other completer UIs out there (not necessarily in Python land) that take a similar approach.

I have no idea how hard that would be to implement, so it may or may not actually not be a tenable solution.

[Carreau]

Thanks Aaron for linking this other issue.

I do (some) of that in IPython already to get the completions types and signature.

I compute the list of completions and turned it into a iterator with a budget of time to compute the types/signature, and once the budget of time is out, use fill values. It is usually enough for the first few completions items to have all informations.