SBML TestSuite semantic/01592 takes 15-25 minutes to run

[matthiaskoenig]

The test takes much too long to run. This creates a ton of problems. E.g. very long travis runtimes and also very long runtimes for the local tests. Tests should normally run < 100ms, upper bound is something of 1-5 seconds for very large and complicated tests. But minutes is too long.

This is most likely due to the delay in the model.

[hemilpanchiwala]

@matthiaskoenig, the test case 1287 from SBML Test Suite, also has a similar problem.

[draeger]

What are the reasons for the long run time? Are particular SBML elements / attributes used?

[hemilpanchiwala]

The description of model 01592 test case from the SBML Test Suite:

category: Test synopsis: Competing events with the same priority, jointly causing a parameter to monotonically increase, checking to make sure the two events are not exactly evenly distributed. NOTE: STOCHASTIC TEST. Your software may fail periodically; it is only supposed to succeed in the majority of cases.

This model contains two events with the same trigger, the same effective priority, both set 'persistent=false', and both of which disable the trigger of the other. This means that every .01 seconds, one fires and the other does not, at random, and increases the parameters Q or R, respectively. A third parameter, S, is assigned the value of Q+R, meaning that it doesn't matter which one fires; S will increase monotonically. A final parameter, 'error' checks to make sure neither Q nor R are chosen more frequently than the other--if the difference gets higher than 0.2, it triggers. Note: The 'errorLow' and 'errorHigh' parameters are a stochastic test, and may not always remain at '0' for all runs. If your software fails, try running it again with a new random number seed, and it may succeed. The values were chosen to be reasonable in the vast majority of cases, but still high enough to reveal problems in software that tends to pick both events either evenly or in an unbalanced manner. This test is a repeat and synthesis of tests 952 and 962, with the additional wrinkle that a delay is used in the assignments.

[draeger]

@hemilpanchiwala Could it also help here if we identify some successful seed value to speed up the test, i.e., does it explicitly use a random number generator?

[hemilpanchiwala]

@draeger, do we use any seed in the simulation for SBML tests🤔?

[draeger]

I think not, but it was a question from my side - maybe we should discuss this. How do we randomly decide which event to pick if they are of equal priority? Are random numbers involved or are they in a randomized data structure (such as a Set), or do we deterministically always pick the first in a list?

[hemilpanchiwala]

Yes, it just clicked to me that there is a method pickRandomEvent() in EquationSystem class. Maybe that can help. Link: https://github.com/hemilpanchiwala/SBSCL/blob/fern/src/main/java/org/simulator/sbml/EquationSystem.java#L1726

[hemilpanchiwala]

Also, this class RNG which is accessed in pickRandomEvent() has variable random_seed. https://github.com/hemilpanchiwala/SBSCL/blob/fern/src/main/java/org/simulator/math/RNG.java

[draeger]

See, there is our random number.

[hemilpanchiwala]

Yes, but it is accessed only once. I mean seed value is set only once in the simulation.