search

ibisngs / knime4ngs-src

Knime4NGS is a set of custom KNIME nodes, designed for processing next generation sequencing datasets.
http://ibisngs.github.io/knime4ngs
GNU General Public License v3.0
5 stars 0 forks source link

Possible Bug: File handles not closed properly? #4

Open AaronNHart opened 5 years ago

AaronNHart commented 5 years ago

Hello,

First of all, thanks again for providing a very useful tool!

I am currently using the STAR node to do some alignments in a long running process. It appears that file handles in this node, do not appear to be closed. The result of this is that my tmp folder is filling up, which is breaking my system.

You can test this my running the STAR node, and then without closing KNIME, running something like:

lsof -a +L1 /<parent-of-tmp>/

Which on my system shows exactly the files referenced by the STAR node, marked as deleted. You can see on the fs that they are no longer there, but df shows they are still occupying the file system. Do you see the same behavior?

Cheers,

Aaron

AaronNHart commented 5 years ago

It looks to me like maybe IO.getFilesOfFolder() is not calling File.close().

hastreiter commented 5 years ago

Hi Aaron,

thanks for your feedback! Nice to hear that you are still using our nodes.

Based on your description, I guess it seems to be related to FileUtils.deleteDirectory(f) which deletes the STAR tmp dir after execution. This method provided by the org.apache.commons.io internally uses deleteOnExit() which schedules a file to be deleted when JVM exits. Therefore I would assume that all files (and file handles) will be deleted as soon as you close the KNIME application. Can you confirm this?

We will try to find a solution but since we are currently not working on this project, it might take a bit of time. Nevertheless we try to fix it as soon as possible!

Kind regards Max

AaronNHart commented 5 years ago

Hi Max,

Thanks for the speedy feedback, especially on a project that isn't the main focus of your attention these days.

The behavior I see is outlined as follows:

du

Shows that the files are not consuming any disk space.

df

shows that the disk is still full, but this looks only at the file system metadata.

Definitely when I restart the KNIME process, the handles are released and everything returns to normal.