r-dbi / RPostgres

A DBI-compliant interface to PostgreSQL
https://rpostgres.r-dbi.org
Other
334 stars 78 forks source link

Are limitations of select() hurting us? #397

Open krlmlr opened 2 years ago

krlmlr commented 2 years ago

@zozlak: In https://man7.org/linux/man-pages/man2/select.2.html I read:

WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applications should instead use poll(2) or epoll(7), which do not suffer this limitation.

Are we affected here?

zozlak commented 2 years ago

Long story short:

Why we have used select() in the first place?

I would guess the original reason for using select() was portability. epoll() is linux-exclusive and poll() is not available on Windows.

Does it affect us - details

The actual question is "is it safe to assume that the socket descriptor number will be always lower than 1024?".

Socket descriptors are allocated by the socket() [and its man page]() says:

The file descriptor returned by a successful call will be the lowest-numbered file descriptor not currently open for the process.

Not bad as it means until our process has more than 1024 files and network sockets opened at the same time, we are on the safe side and the limit of 1024 seems pretty safe.

Moreover, the default limit of opened files for normal users in a few linux systems I checked (my laptop with ubuntu 22.04, server at work with centos 7) is also 1024 so the process would anyway fail trying to open more of them (which would cause database connection error as the socket() call made by the PQconnectdbParams() internals would fail).

That being said if someone has higher max opened files ulimit and opens more than 1024 files and sockets in one process, (s)he will be affected by the limit you described.