ValueError: filedescriptor out of range during test_subprocess on NetBSD

[furkanonder]

Bug report

Bug description:

home$ ./python -m test test_subprocess -m test_no_leaking

Output:

Using random seed: 2548767682
0:00:00 load avg: 0.02 Run 1 test sequentially in a single process
0:00:00 load avg: 0.02 [1/1] test_subprocess
/home/blue/cpython/Lib/subprocess.py:1136: ResourceWarning: subprocess 16019 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/blue/cpython/Lib/test/support/__init__.py:816: ResourceWarning: unclosed file <_io.FileIO name=1015 mode='wb' closefd=True>
  gc.collect()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/blue/cpython/Lib/test/support/__init__.py:816: ResourceWarning: unclosed file <_io.FileIO name=1016 mode='rb' closefd=True>
  gc.collect()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/blue/cpython/Lib/test/support/__init__.py:816: ResourceWarning: unclosed file <_io.FileIO name=1018 mode='rb' closefd=True>
  gc.collect()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test test_subprocess failed -- Traceback (most recent call last):
  File "/home/blue/cpython/Lib/test/test_subprocess.py", line 1247, in test_no_leaking
    data = p.communicate(b"lime")[0]
           ~~~~~~~~~~~~~^^^^^^^^^
  File "/home/blue/cpython/Lib/subprocess.py", line 1218, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/blue/cpython/Lib/subprocess.py", line 2121, in _communicate
    ready = selector.select(timeout)
  File "/home/blue/cpython/Lib/selectors.py", line 314, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
              ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: filedescriptor out of range in select()

test_subprocess failed (1 error)

== Tests result: FAILURE ==

1 test failed:
    test_subprocess

Total duration: 1.1 sec
Total tests: run=2 (filtered)
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE
home$

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

• gh-124987

[furkanonder]

home$ cat fd_count.c
#include <stdio.h>
#include <sys/select.h>

int main() {
    printf("FD_SETSIZE: %d\n", FD_SETSIZE);
    return 0;
}
home$ gcc fd_count.c -o fd_count.o
home$ ./fd_count.o
FD_SETSIZE: 256
home$ ulimit -Sn
1024
home$ ulimit -Hn
3404
home$ ulimit -n
1024

I don't think there is a leak, the test was successful when I increased the number of file descriptors with ulimit.

home# ulimit -n 2048
home# ./python -m test test_subprocess -m test_no_leaking
Using random seed: 389439817
0:00:00 load avg: 0.11 Run 1 test sequentially in a single process
0:00:00 load avg: 0.11 [1/1] test_subprocess

== Tests result: SUCCESS ==

1 test OK.

Total duration: 467 ms
Total tests: run=2 (filtered) skipped=2
Total test files: run=1/1 (filtered)
Result: SUCCESS
home#

I can set the max_handles=1020 as final value, above the 1020 the test is failing. In the system FD_SETSIZE is equal to 256. That's why I thought to set max_handles as 2 times the FD_SETSIZE value in the test.