glaciers-in-archives / snowman

A static site generator for SPARQL backends.
GNU Lesser General Public License v3.0
112 stars 11 forks source link

UNIX: Build crash caused by having to many cache files #53

Closed Abbe98 closed 1 year ago

Abbe98 commented 1 year ago

Unix systems put a limit on how many files that can be kept open at any given time. Snowman on the other hand tries to read all of the existing cache files before building a project causing certain large projects to fail. Snowman should close these files correctly and only keep their contents in memory.

An alternative to just fixing the bug itself is to make Snowman only read cache files when needed and keep them in memory after that moment, this is especially useful in situations where you don't want to invalidate unused cache items, wouldn't slow down the build, and could save some memory for users.