Possible memory leak on 2020-04-06 version of owltools

[kshefchek]

When testing out https://github.com/owlcollab/owltools/releases/tag/2020-04-06 running export OWLTOOLS_MEMORY=45G && owltools /data/all.owl --use-fsim --sim-load-lcs-cache /data/owlsim.cache --sim-load-ic-cache /data/ic-cache.owl --start-sim-server -p 9031

with the files here: https://archive.monarchinitiative.org/202003/owlsim/

This loads for about 45 minutes then runs out of memory, I bumped the memory to 55G and it seems to get suck on the initial load (killed it after an hour)

As a reference point the 0.3.0 version takes 12 minutes to load with 45G. Running this with an openjdk-8-slim docker image

[matentzn]

Depending on what your source is (all.owl), I can tell you that I have struggled getting that to pass through (full uPheno, remember this). But in your case, the exact same ontology as input has so extremely different runtimes depending on the version? That seems odd! I dont think the code of owltools has been updated in a long time!

Also, I dont think there is any hope of anyone fixing owltools in any major ways at this point; most of its functionality has moved to ROBOT, and stuff like this here (similarity scores) are supposed to be implemented as part of phenol; we were banking on the renewal to the money to do this, so now this is on ice. In any case, in my experience bumping the memory to something like 120 GB and pruning the input ontology down the bare minimum should work. Do you have any hypothesis what could cause this if 1) the code has not changed and 2) the java version has not changed?

[balhoff]

There have been changes between those two versions: https://github.com/owlcollab/owltools/compare/v0.3.0...2020-04-06

Assuming Kent is actually talking about the v0.3.0 tag—but I think owltools has had its version set to 0.3.0 throughout that period.

[matentzn]

Seems to that the changes didnt have anything to do with the similarity implementation though; The only thing I can see is that all the System.out.println() were replaced by LOG.debug().. cant imagine that causing an issue.

[balhoff]

I wonder about this one: https://github.com/owlcollab/owltools/commit/4c1326f27e2d07667a173371f641918de45fc7b7

[matentzn]

what a commit.. huuuuuge. Thats why i am not a fan of blind squash and merge... So you think it could be due to the reasoner and this "deep copy" thing? In any case, its a done deal now; if we have to fix this, we have to actually sit down and debug the code I think.

[kshefchek]

a background on this, I'm dockerizing our services and realized our owlsim version on production is from 2016, but if nothing major has happened the 0.3.0 release in github works just as well

[matentzn]

In that case I would use the old version while we work on replacing the phenodigm implementation altogether as part of the renewal grant...

[kshefchek]

we can close then, it doesn't sound like a memory leak just a refactor that puts more into memory via deep copies of objects