Blocks of consecutive tests fail with "too many open files" ... looks like file handle leaks

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Build with mock on Fedora 15 (unfortunately the issue is not systematically 
reproducible - it appears on few computers only - probably depends on their 
ulimit configuration? not sure)

What is the expected output? 
No errors.

What do you see instead?
Look at the attachaments ... 1.) workstation 2.) build server

What version of SnakeYAML are you using? On what Java version?
1.8

Original issue reported on code.google.com by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 9:49

Attachments:

• build_log_1.txt
• build_log_2.txt

[GoogleCodeExporter]

The problem is not related to SnakeYAML. It can indicate a bug in Java or a 
configuration problem for the operating system. There is nothing SnakeYAML can 
do to solve the problem.

Original comment by py4fun@gmail.com on 9 Jun 2011 at 10:15

[GoogleCodeExporter]

I think it needs more investigation.
We may leak file handles because of not closing input streams. Brief look gives 
- seems like we do not close input streams...  :(

But as I said - need more investigation.

Original comment by alexande...@gmail.com on 9 Jun 2011 at 10:27

[GoogleCodeExporter]

Hi py4fun.

Thanks for the answer.
Are You really sure, that this can't be caused by forgotten close calls?
As forgotten close calls have exactly this kind of symptoms.
File handles can get depleted when there's a missing close call in methods 
which are called multiple times and the differencies in the behavior are 
usually 
caused by race conditions related to the current system state (number of 
concurrent processes, etc.)

Please, try to reconsider that.
Thank You.

Regards,
Jaromir.

Original comment by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 10:35

[GoogleCodeExporter]

I'm gonna try the findbugs tool in order to find possible leaks and send 
patches if I find any ... 

Original comment by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 11:00

[GoogleCodeExporter]

The following methods were marked as suspicious for failing to clean up stream 
or resources:

examples.LoadExampleTest.testLoadFromStream()
examples.LoadExampleTest.testLoadManyDocuments()
org.yaml.snakeyaml.issue99.YamlBase64Test.testLocalBinaryTag()
org.yaml.snakeyaml.issue99.YamlBase64Test.testRedefineBinaryTag()
org.yaml.snakeyaml.issue99.YamlBase64Test.testYamlBase64Loading()
org.yaml.snakeyaml.issue99.YamlBase64Test.testYamlBase64LoadingLiteral()

Original comment by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 11:20

[GoogleCodeExporter]

We are now closing more input streams. All the mentioned and many more others 
are now closing within the test.
Give it a try.

Original comment by py4fun@gmail.com on 9 Jun 2011 at 11:35

• Changed state: Started

[GoogleCodeExporter]

Thank You.

I'm going to merge the following two commits into a patch file against 1.8 and 
test these changes on our build system.

I'm unsure if they can be applied directly on the 1.8 since there could be more 
changes dependent on changes in different files which won't be propagated with 
this patch.

----
4d1e0ed27f01:

/src/test/java/examples/LoadExampleTest.java
/src/test/java/org/yaml/snakeyaml/issues/issue99/YamlBase64Test.java    

a4b407aba3a5:

/src/changes/changes.xml
/src/test/java/org/pyyaml/AnInstance.java
/src/test/java/org/pyyaml/PyCanonicalTest.java
/src/test/java/org/pyyaml/PyEmitterTest.java
/src/test/java/org/pyyaml/PyErrorsTest.java
/src/test/java/org/pyyaml/PyImportTest.java
/src/test/java/org/pyyaml/PyReaderTest.java
/src/test/java/org/pyyaml/PyStructureTest.java
/src/test/java/org/pyyaml/PyTokensTest.java
----

Original comment by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 12:14

[GoogleCodeExporter]

The following patch (including changes from commits 4d1e0ed27f01 & 
a4b407aba3a5) against the 1.8 solved the problem. 

http://jcapik.fedorapeople.org/files/snakeyaml/4/snakeyaml-issue121-file-handle-
leaks.patch

The package has been build successfully.

I believe this issue can be marked as fixed.

Original comment by jcapik%r...@gtempaccount.com on 9 Jun 2011 at 3:11

[GoogleCodeExporter]

Original comment by py4fun@gmail.com on 9 Jun 2011 at 3:33

• Changed state: Fixed