Update the SBMLTestSuite Runner for L3V2

draeger-lab / SBSCL

The Systems Biology Simulation Core Library (SBSCL) provides an efficient and exhaustive Java implementation of methods to interpret the content of models encoded in the Systems Biology Markup Language (SBML) and its numerical solution.

https://draeger-lab.github.io/SBSCL/

GNU Lesser General Public License v3.0

20 stars 13 forks source link

Update the SBMLTestSuite Runner for L3V2 #35

Closed matthiaskoenig closed 4 years ago

matthiaskoenig commented 4 years ago

Update the SBMLTestSuiteRunner

[x] to use latest test files
[x] include SBML L3V2 tests
[x] create parametrized tests (for every single test in the test runner to have a single test case)
[x] refactor the test runner so collection of tests and running of a single test are separated
[x] fail tests if they take > 60 seconds
[ ] allow to run test suite in parallel

see SBMLTestSuiteRunner, download_sbml-test-suite.sh

matthiaskoenig commented 4 years ago

Also could you look into running the test-suite in parallel. This takes very long at the moment and would be great if one could run these tests in parallel via a maven flag. The tests do not share any state and have no raise conditions so no issues should arise in parallelization. See for instance https://www.baeldung.com/maven-junit-parallel-tests

hemilpanchiwala commented 4 years ago

@matthiaskoenig, Should we make running SBML Test Suite tests parallel as I think this will create issues with the GLPKSolver (which we observed even before)?

matthiaskoenig commented 4 years ago

This is not a priority. Open a separate issue for the parallelization where you document the issues with the parallel running of the FBA tests. You can close this one here. Like mentioned in the discussion, the GLPK can probably not be run in parallel via threads. See http://glpk-java.sourceforge.net/usage.html:

The GLPK library is not thread safe. Never two threads should be running that access the GLPK library at the same time. When a new thread accesses the library it should call GLPK.glp_free_env(). When using an GlpkTerminalListener it is necessary to register GlpkTerminal again by calling GLPK.glp_term_hook(null, null).

The only reason why this works in the TestRunner is because it starts a separate Java program for every test.

hemilpanchiwala commented 4 years ago

@matthiaskoenig, I am creating a separate issue for this and linking this issue to the current PR, so can be closed on its merge.