ncbo / ontologies_linked_data

Models and serializers for ontologies and related artifacts backed by 4store
Other
18 stars 8 forks source link

Zlib::GzipWriter errors during test runs #107

Closed mdorf closed 4 years ago

mdorf commented 4 years ago

I see sporadic Zlib::GzipWriter errors that appear during test runs. You can reproduce it by running the test_ontology_submission.rb tests:

bundle exec ruby ./test/run_tests.rb -f ./test/models/test_ontology_submission.rb

The full error output is:

zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.
zlib(finalizer): the stream was freed prematurely.

Example:

TestOntologySubmission#test_submission_parse = 89.26 s = .
TestOntologySubmission#test_custom_property_generation = 3.53 s = .
TestOntologySubmission#test_submission_diff_across_ontologies = zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.
zlib(finalizer): the stream was freed prematurely.
11.36 s = .
TestOntologySubmission#test_sanity_check_zip = 3.47 s = .
TestOntologySubmission#test_valid_ontology = 0.37 s = .
TestOntologySubmission#test_custom_obsolete_property = 3.62 s = .
TestOntologySubmission#test_duplicated_file_names = 0.19 s = .
TestOntologySubmission#test_index_properties = zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.
zlib(finalizer): the stream was freed prematurely.
5.72 s = .
TestOntologySubmission#test_submission_root_classes = 6.86 s = .
TestOntologySubmission#test_submission_parse_subfolders_zip = 4.83 s = .
TestOntologySubmission#test_semantic_types = 1.28 s = .
TestOntologySubmission#test_process_submission_archive = zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.
zlib(finalizer): the stream was freed prematurely.
85.09 s = .
mdorf commented 4 years ago

This error affects both 4store and AllegroGraph.

mdorf commented 4 years ago

I believe the problem is caused by this code in index method of ontology_submission.rb: https://github.com/ncbo/ontologies_linked_data/blob/master/lib/ontologies_linked_data/models/ontology_submission.rb#L1153

csv_writer = LinkedData::Utils::OntologyCSVWriter.new
csv_writer.open(self.ontology, self.csv_path)

csv_writer.open runs:

@file = File.new(path, 'w')
@gz = Zlib::GzipWriter.new(@file)

Eventually, there is a call to csv_writer.close, but if there is an earlier exception thrown, this code never executes: https://github.com/ncbo/ontologies_linked_data/blob/master/lib/ontologies_linked_data/models/ontology_submission.rb#L1265

I believe this could result in a memory leak.

mdorf commented 4 years ago

Added an explicit rescue clause that calls csv_writer.close if an exception occurs. This has fixed the issue.

mdorf commented 4 years ago

fixed