"Too many files open" during API requests to clear cache

[bbpennel]

We have an automated workflow that makes requests to the "PurgeItemFromCache" API endpoint in cantaloupe 5.0.5 after a jp2 gets replaced in our system, so that all of its derivatives in cantaloupe's cache will get regenerated. But we've had cantaloupe run out of file handles repeatedly while regenerating a set of 5700 jp2s.

I'm able to replicate the issue pretty easily in our system by making concurrent calls to "PurgeItemFromCache" with a populated cache. If I monitor the file handles held by the process, I can watch them quickly pile up and never be released. They are basically all references to empty directories, like:

lr-x------ 1 root cantaloupe 64 Oct 16 11:50 100 -> /opt/cantaloupe-5.0.5/cache/image/06/6c/55
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1000 -> /opt/cantaloupe-5.0.5/cache/image/a4/49/f7
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1001 -> /opt/cantaloupe-5.0.5/cache/image/a4/49/f7
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1002 -> /opt/cantaloupe-5.0.5/cache/image/af/d2/05
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1003 -> /opt/cantaloupe-5.0.5/cache/image/af/d2/05
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1004 -> /opt/cantaloupe-5.0.5/cache/image/af/d2/05
lr-x------ 1 root cantaloupe 64 Oct 16 11:50 1005 -> /opt/cantaloupe-5.0.5/cache/image/af/d2/05

I don't know if the same issue occurs with a request to purge the whole cache, or when the background cache clearing processes run. This is occurring in a RHEL7 server. I don't see any similar behaviors come from read requests to cantaloupe, like requesting tiles or thumbnails.

[bbpennel]

I've opened a PR which seems to fix the issue on our local servers. I've done several hundred purge requests without any new file handles being added.

[DiegoPino]

Fixed via https://github.com/cantaloupe-project/cantaloupe/commit/0dccf299b9310f2293e6cb467799a5345fd58e39 Thanks for your contribution @bbpennel!