Error while running SDevo through the command line

[mkyriak]

Hello!

I am running SDevo on the command line with the following command:

java -jar SDevo.jar test_state_clocks_template.xml

I am using the xml template from here. While I tested in on the sequences from Li et al (the sequences used in the manuscript) it seemed to work fine. However, when I replaced the sequences with mine, along with the loc (loc0 or loc1) for each sequence (depending if it is found on edge or in the center of the tumor), SDevo run crashes. The sequences I am using contain gaps and I generated the multiple sequence alignment with vcf2msa.py.

This is the error I am getting:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at org.apache.commons.math3.ode.sampling.AbstractStepInterpolator.<init>(AbstractStepInterpolator.java:188) at org.apache.commons.math3.ode.nonstiff.RungeKuttaStepInterpolator.<init>(RungeKuttaStepInterpolator.java:85) at org.apache.commons.math3.ode.nonstiff.DormandPrince54StepInterpolator.<init>(DormandPrince54StepInterpolator.java:117) at org.apache.commons.math3.ode.nonstiff.DormandPrince54StepInterpolator.doCopy(DormandPrince54StepInterpolator.java:142) at org.apache.commons.math3.ode.sampling.AbstractStepInterpolator.copy(AbstractStepInterpolator.java:272) at org.apache.commons.math3.ode.ContinuousOutputModel.handleStep(ContinuousOutputModel.java:197) at org.apache.commons.math3.ode.AbstractIntegrator.acceptStep(AbstractIntegrator.java:396) at org.apache.commons.math3.ode.nonstiff.EmbeddedRungeKuttaIntegrator.integrate(EmbeddedRungeKuttaIntegrator.java:303) at org.apache.commons.math3.ode.AbstractIntegrator.integrate(AbstractIntegrator.java:228) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:378) at sdevo.bdmm.FlatTypeMappedTree.getInternalState(FlatTypeMappedTree.java:502) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:351) at sdevo.bdmm.FlatTypeMappedTree.getInternalState(FlatTypeMappedTree.java:501) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:351) at sdevo.bdmm.FlatTypeMappedTree.doStochasticMapping(FlatTypeMappedTree.java:173) at sdevo.bdmm.BdmmStateClock.getRateForBranch(BdmmStateClock.java:69) at beast.evolution.likelihood.BeagleTreeLikelihood.traverse(BeagleTreeLikelihood.java:928) at beast.evolution.likelihood.BeagleTreeLikelihood.traverse(BeagleTreeLikelihood.java:964) at beast.evolution.likelihood.BeagleTreeLikelihood.calculateLogP(BeagleTreeLikelihood.java:689) at beast.evolution.likelihood.TreeLikelihood.calculateLogP(TreeLikelihood.java:373) at beast.evolution.likelihood.ThreadedTreeLikelihood.calculateLogPByBeagle(ThreadedTreeLikelihood.java:307) at beast.evolution.likelihood.ThreadedTreeLikelihood.calculateLogP(ThreadedTreeLikelihood.java:269) at beast.core.util.CompoundDistribution.calculateLogP(CompoundDistribution.java:114) at beast.core.util.CompoundDistribution.calculateLogP(CompoundDistribution.java:114) at beast.core.MCMC.propagateState(MCMC.java:544) at beast.core.MCMC.doLoop(MCMC.java:422) at beast.core.MCMC.run(MCMC.java:367) at beast.app.BeastMCMC.run(BeastMCMC.java:382) at beast.app.BeastMCMC.main(BeastMCMC.java:583)

I tried increasing the memory to java -Xmx350g -jar but it still gives me the error above.

Thank you in advance for your time and kind help!

[mkyriak]

Hello @nicfel ! Could you please have a look at this error? Thank you!

[nicfel]

Hi @mkyriak, What happens before that? Are there a few iterations being logged before, or does it crash when initializing?

[mkyriak]

Hi @nicfel ,

Thank you for the reply!

I get the following before: 443000 -26031.6477 -25981.8474 -49.8002 5m29s/Msamples 444000 -26030.9092 -25982.5075 -48.4017 5m29s/Msamples 445000 -26030.3889 -25980.4997 -49.8892 5m29s/Msamples 446000 -26024.9490 -25980.3650 -44.5839 5m29s/Msamples 447000 -26025.9651 -25980.4066 -45.5584 5m29s/Msamples 448000 -26034.5876 -25982.1268 -52.4607 5m29s/Msamples 449000 -26033.4778 -25980.3319 -53.1458 5m30s/Msamples 450000 -26031.0989 -25985.3211 -45.7778 5m30s/Msamples 451000 -26035.1901 -25990.4449 -44.7452 5m30s/Msamples 452000 -26024.4349 -25981.0009 -43.4339 5m29s/Msamples 453000 -26025.2984 -25980.5450 -44.7533 5m29s/Msamples 454000 -26027.5705 -25979.8745 -47.6959 5m30s/Msamples 455000 -26025.6729 -25980.3543 -45.3186 5m30s/Msamples 456000 -26027.6772 -25981.6388 -46.0384 5m30s/Msamples 457000 -26034.9131 -25985.9127 -49.0003 5m30s/Msamples 458000 -26025.0298 -25980.0591 -44.9707 5m30s/Msamples 459000 -26027.2317 -25982.9240 -44.3077 5m30s/Msamples 460000 -26025.8174 -25982.3925 -43.4249 5m30s/Msamples Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at org.apache.commons.math3.ode.sampling.AbstractStepInterpolator.<init>(AbstractStepInterpolator.java:190) at org.apache.commons.math3.ode.nonstiff.RungeKuttaStepInterpolator.<init>(RungeKuttaStepInterpolator.java:85) at org.apache.commons.math3.ode.nonstiff.DormandPrince54StepInterpolator.<init>(DormandPrince54StepInterpolator.java:117) at org.apache.commons.math3.ode.nonstiff.DormandPrince54StepInterpolator.doCopy(DormandPrince54StepInterpolator.java:142) at org.apache.commons.math3.ode.sampling.AbstractStepInterpolator.copy(AbstractStepInterpolator.java:272) at org.apache.commons.math3.ode.ContinuousOutputModel.handleStep(ContinuousOutputModel.java:197) at org.apache.commons.math3.ode.AbstractIntegrator.acceptStep(AbstractIntegrator.java:396) at org.apache.commons.math3.ode.nonstiff.EmbeddedRungeKuttaIntegrator.integrate(EmbeddedRungeKuttaIntegrator.java:303) at org.apache.commons.math3.ode.AbstractIntegrator.integrate(AbstractIntegrator.java:228) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:378) at sdevo.bdmm.FlatTypeMappedTree.getInternalState(FlatTypeMappedTree.java:502) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:351) at sdevo.bdmm.FlatTypeMappedTree.getInternalState(FlatTypeMappedTree.java:502) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:351) at sdevo.bdmm.FlatTypeMappedTree.getInternalState(FlatTypeMappedTree.java:501) at sdevo.bdmm.FlatTypeMappedTree.backwardsIntegrateSubtree(FlatTypeMappedTree.java:351) at sdevo.bdmm.FlatTypeMappedTree.doStochasticMapping(FlatTypeMappedTree.java:173) at sdevo.bdmm.BdmmStateClock.getRateForBranch(BdmmStateClock.java:69) at beast.evolution.likelihood.BeagleTreeLikelihood.traverse(BeagleTreeLikelihood.java:928) at beast.evolution.likelihood.BeagleTreeLikelihood.traverse(BeagleTreeLikelihood.java:964) at beast.evolution.likelihood.BeagleTreeLikelihood.calculateLogP(BeagleTreeLikelihood.java:689) at beast.evolution.likelihood.TreeLikelihood.calculateLogP(TreeLikelihood.java:373) at beast.evolution.likelihood.ThreadedTreeLikelihood.calculateLogPByBeagle(ThreadedTreeLikelihood.java:307) at beast.evolution.likelihood.ThreadedTreeLikelihood.calculateLogP(ThreadedTreeLikelihood.java:269) at beast.core.util.CompoundDistribution.calculateLogP(CompoundDistribution.java:114) at beast.core.util.CompoundDistribution.calculateLogP(CompoundDistribution.java:114) at beast.core.MCMC.propagateState(MCMC.java:544) at beast.core.MCMC.doLoop(MCMC.java:422) at beast.core.MCMC.run(MCMC.java:367) at beast.app.BeastMCMC.run(BeastMCMC.java:382) at beast.app.BeastMCMC.main(BeastMCMC.java:583) [SDevo.log](https://github.com/nicfel/SDevo/files/11973559/SDevo.log)

I have also attached the error file here.

Kindest regards, Maria

[nicfel]

My guess is that some of the parameters go to very extreme values (e.g. the tree height or origin goes to infinity or the birth rates go to infinity, or just very high). If the values get too high, then the ODE solving needs to save too many intermediate steps into memory causing the code to crash. If that is the issue, changing priors or limiting upper values on parameters could solve it.

[mkyriak]

Thank you, I tried changing the value for clockRate, but it didn't help. I will try setting up an upper limit for all the parameters and see how it goes.

[mkyriak]

Hello @nicfel , I left this a bit, but I am back to it. I changed the priors' values and set up upper limits for the parameters, however, I am getting this error now:

6167000 -9838.5177 -9836.1353 -2.3823 9m38s/Msamples 6168000 -9835.6791 -9833.0389 -2.6402 9m38s/Msamples 6169000 -9839.0698 -9833.8270 -5.2427 9m38s/Msamples 6170000 -9838.5555 -9833.4606 -5.0948 9m38s/Msamples P(posterior) = -9859.964387782255 (was -9838.555530309619) ** P(prior) = -5.094832334136447 (was -5.094832334136447) P(BDMMPrime.t:state_dependent_clock_sim) = -68.14961257042178 (was -68.14961257042178) P(birthRateCanonicalPrior.t:state_dependent_clock_sim) = -3.393951235487097 (was -3.393951235487097) P(HyperPrior.hyperLogNormalDistributionMbirthRate.t:state_dependent_clock_sim) = 76.7017855123741 (was 76.7017855123741) P(deathRateCanonicalPrior.t:state_dependent_clock_sim) = -1.2337872562645509 (was -1.2337872562645509) P(migrationRateCanonicalPrior.t:state_dependent_clock_sim) = -0.16560135629967232 (was -0.16560135629967232) P(originBDMMPrimePrior.t:state_dependent_clock_sim) = -8.85366542803745 (was -8.85366542803745) P(likelihood) = -9854.869555448118 (was -9833.460697975483) ** P(treeLikelihood.state_dependent_clock_sim) = -9854.869555448118 (was -9854.869555448118) At sample 6170000 Likelihood incorrectly calculated: -9838.555530309619 != -9859.964387782255(21.408857472635646) Operator: SAUniform(BDMMPrimeSAUniformOperator.t:state_dependent_clock_sim) Too many corrections. There is something seriously wrong that cannot be corrected

I tried to change the weight for BDMMPrimeSAUniformOperator parameter but it gives me the initial java error.

Do you have any idea what is causing this? Also, is it a bad idea to comment this parameter?

Thank you once again, Maria