search

RTXteam / RTX

Software repo for Team Expander Agent (Oregon State U., Institute for Systems Biology, and Penn State U.)
https://arax.ncats.io/
MIT License
33 stars 21 forks source link

out of memory error when running multi_owl_to_json_kg.py with full owl-load-inventory.yaml #596

Closed amykglen closed 4 years ago

amykglen commented 4 years ago

getting this error when running multi_owl_to_json_kg.py on kg2dev:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.base/java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:86)
    at java.base/java.lang.StringBuilder.<init>(StringBuilder.java:112)
    at com.fasterxml.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:346)
    at com.fasterxml.jackson.core.io.SegmentedStringWriter.getAndClear(SegmentedStringWriter.java:83)
    at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:1037)
    at org.geneontology.obographs.io.OgJsonGenerator.prettyJsonString(OgJsonGenerator.java:18)
    at org.geneontology.obographs.io.OgJsonGenerator.render(OgJsonGenerator.java:11)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:267)
    at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:209)
    at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:3712)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
    at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
    at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Traceback (most recent call last):
  File "/home/ubuntu/kg2-code/multi_owl_to_json_kg.py", line 960, in <module>
    test_mode)
  File "/home/ubuntu/kg2-code/multi_owl_to_json_kg.py", line 193, in make_kg2
    ont_source_info_dict['title'])
  File "/home/ubuntu/kg2-code/multi_owl_to_json_kg.py", line 130, in load_owl_file_return_ontology_and_metadata
    ontology = make_ontology_from_local_file(file_name)
  File "/home/ubuntu/kg2-code/multi_owl_to_json_kg.py", line 110, in make_ontology_from_local_file
    assert cp.returncode == 0
AssertionError

occurred while reading umls-msh.ttl - this was the last stdout line:

Reading ontology file: umls-msh.ttl; size: 634784.30 KiB
amykglen commented 4 years ago

problem is likely just that I hadn't set owltools memory before running the script - can do so using these lines (as they appear in build-multi-owl-kg.sh):

MEM_GB=`${CODE_DIR}/get-system-memory-gb.sh`
export OWLTOOLS_MEMORY=${MEM_GB}G

(trying this out now... thanks, Steve!)

amykglen commented 4 years ago

issue resolved