man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
http://arcticdb.io
Other
1.52k stars 93 forks source link

Make library manager an LRU cache #1930

Closed willdealtry closed 3 weeks ago

willdealtry commented 1 month ago

At the moment as we open more and more libraries we never close them, meaning that we can run out of filehandles.

poodlewars commented 1 month ago

Will this help with the file descriptor exhaustion problem? Given https://github.com/man-group/ArcticDB/pull/1908 aren't we going to hold them open regardless of this change?