'RuntimeError: async generator ignored GeneratorExit' when ThreadedCompleter-> NestedCompleter->WordCompleter is given a large number of completions to asynchronously parse

[Tom-Goring]

I'm developing a REPL where one of the options has a large number of completions (a few thousand). Each time a character is inputted or a completion is chosen a Runtime error is raised somewhere in a generator and ignored - spawning this message into stdout (each error is a keypress):

The completer is a NestedCompleter ending in a WordCompleter (which is the one with many completions) and complete_in_thread=True. The prompt being used is prompt_async(). Prompt_toolkit version 3.0.25.

Is there something I can do as a caller to avoid this beyond reducing the number of completions, or is an underlying change required somewhere?

[jonathanslenders]

Hi Tom,

Thank you for reporting the issue! Would it be possible to create a small script to reproduce the issue?

I created the following script, which creates 100k completions without issues:

#!/usr/bin/env python
from prompt_toolkit import prompt
from prompt_toolkit.completion import WordCompleter
import random
import string

def generate_word():
    return "".join(random.choice(string.ascii_lowercase) for i in range(10))

completer = WordCompleter([generate_word() for i in range(100000)], ignore_case=True)

def main():
    text = prompt("Type something: ", completer=completer, complete_while_typing=True)
    print("You said: %s" % text)

if __name__ == "__main__":
    main()

This script however does not yet use NestedCompleter.

[Tom-Goring]

Hey there - admittedly I don't know much about async in python so I'm not sure where the problem is, but it seems to be related to our use of nest_asyncio. The following script seems to provoke the error if I type relatively quickly:

#!/usr/bin/env python
from prompt_toolkit import PromptSession
from prompt_toolkit.completion import WordCompleter
import random
import string
import asyncio
import nest_asyncio

nest_asyncio.apply()

def generate_word():
    return "".join(random.choice(string.ascii_lowercase) for i in range(10))

completer = WordCompleter([generate_word() for i in range(100000)], ignore_case=True)

session = PromptSession(
    "> ",
    completer=completer,
    complete_in_thread=True,
    refresh_interval=1,
)

async def main():
    text = await session.prompt_async()
    print("You said: %s" % text)

if __name__ == "__main__":
    asyncio.run(main())