prompt-toolkit / python-prompt-toolkit

Library for building powerful interactive command line applications in Python
https://python-prompt-toolkit.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
9.11k stars 718 forks source link

filedescriptor still out of range in select #1702

Open jstucke opened 1 year ago

jstucke commented 1 year ago

Hi, I'm still having some problems with select.select raising an exception "ValueError: filedescriptor out of range in select()" in src/prompt_toolkit/input/posix_utils.py", line 72 if the file descriptor number is too large (>1023). This happens when passing in a pipe instead of stdin during testing. It seems to be similar to problems that were already discussed in #354.

What makes this especially problematic is that the exception occurs inside the prompt loop which results in the prompt (and pytest) hanging indefinitely. A simple test to reproduce the problem:

import os
from typing import NamedTuple

import pytest

from prompt_toolkit import PromptSession
from prompt_toolkit.input import create_pipe_input
from prompt_toolkit.input.base import PipeInput
from prompt_toolkit.output import DummyOutput

@pytest.fixture
def prompt():
    file_descriptors = []
    try:  # if an "OSError: Too many open files" is thrown, you may need to adjust the fp limit (`ulimit -n`)
        for _ in range(512):  # this should be enough to create a file descriptor > 1023
            read_fd, write_fd = os.pipe()
            file_descriptors.extend([read_fd, write_fd])
        with create_pipe_input() as input_:
            session = PromptSession(input=input_, output=DummyOutput())
            yield Prompt(session, input_)
    finally:
        for fd in file_descriptors:
            os.close(fd)

class Prompt(NamedTuple):
    session: PromptSession
    input: PipeInput

def test_too_many_open_files(prompt):
    prompt.input.send_text(f"foobar\n")
    # the prompt will hang if `select.select` is used because it causes an exception "filedescriptor out of range"
    prompt.session.prompt("foobar")

As discussed in #354, it is possible to easily fix this by replacing select.select with select.poll. I have added a patch which seemingly fixes this: select_poll_patch.txt