search

citymania-org / grf-py

A Python framework for making NewGRFs for OpenTTD
GNU General Public License v2.0
14 stars 3 forks source link

Too many open files when many sprites share sources #32

Closed ahyangyi closed 3 months ago

ahyangyi commented 3 months ago

Hi,

I am adding purchase-specific graphics to my newgrf, which is generated by composition. The process is quite slow, because some of the purchase graphics is composed from hundreds of sprites... hence I need the cache manager to accelerate it.

The problem is that the cache manager now opens too many files simultaneously. I can ulimit -n to temporarily alleviate the problem, but the new bound will probably been hit again not long after.

I know that the current design of the cache manager is to only open source files when necessary, and release the memory aggressively to reduce peak memory consumption. Which is great. However, perhaps fd should be released more aggressively than memory, since the default fd limit is only 1,024, which will be exceeded way before the typical PC's memory limit is reached.

ahyangyi commented 3 months ago

Forgot how I avoided the problem, but for now I no longer need to ulimit. Weird. I'll reopen this when it happens again...

ldpl commented 1 month ago

I fixed a similar issue in BonkyGFX by implementing unload: https://github.com/lukaskrepel/BonkyGFX/commit/36a3c00a2fc77e431dfe828a85ab63c178bcf315