search

OpenModelica / OpenModelica

OpenModelica is an open-source Modelica-based modeling and simulation environment intended for industrial and academic usage.
https://openmodelica.org
Other
818 stars 304 forks source link

FlexibleBeam models in ScalableTestSuite fail with --daeMode because of time event handling #7917

Open casella opened 2 years ago

casella commented 2 years ago

The FlexibleBeam models of the ScalableTestSuite library are currently broken when ran with --daeMode, see e.g. FlexibleBeamModelica_N_2:

assert            | debug   | ##IDA## discrete update failed flag -6!

They once used to work, as reported in this paper, so something went wrong in the meantime, and we missed the regression.

@AnHeuermann, @kabdelhak, could you please check?

AnHeuermann commented 2 years ago

I updated SUNDIALS and therefor how we use IDA. I'll debug a bit and check what's going wrong here.

casella commented 2 years ago

Aha, sure. Can you please remind me when you did the commit, so I can check the regression reports?

I guess this should go in 1.18.1, because otherwise we have regressions in 1.18.0

Thanks!

AnHeuermann commented 2 years ago

That was in https://github.com/OpenModelica/OpenModelica/commit/ba83d81e5f1f132591524d50d9e5c8af5043168e.

AnHeuermann commented 2 years ago

Function IDACalcIC is returning the error code IDA_LSETUP_FAIL which means that the linear solver’s setup function failed in an unrecoverable manner.

Not sure what we are missing. We set up KLU as linear solver and the initialization is done successfully and it fails at the first time step during the setup with SUNLinSolSetup.

casella commented 2 years ago

That was in ba83d81.

That doesn't seem to be the culprit, no such regressions back then.

casella commented 2 years ago

I tried to figure out from this plot, but it's hard, also because some tests are randomly succeeding or failing based on timeouts.

Too bad we don't have regression reports tracking individual models.

I could manually search for FlexibleBeamModelica_N_2 in all the .html reports here, but that would be quite a stupid task and take quite some time.

@adrpo, @sjoelund, @perost, @arun3688, maybe a simple php or javascript snippet could do that?

EDIT: maybe I could just use Google, if that page is accessible to the robot crawlers.

EDIT 2: This Google query should work, but only reports me one page back from 2018

casella commented 2 years ago

Not sure what we are missing. We set up KLU as linear solver and the initialization is done successfully and it fails at the first time step during the setup with SUNLinSolSetup.

As far as I understand, normally "solver setup failed" really means "singular model". Possibly because of some error in computing the sparse Jacobian, e.g. missing some elements of it.

sjoelund commented 2 years ago

I could manually search for FlexibleBeamModelica_N_2 in all the .html reports here, but that would be quite a stupid task and take quite some time.

$ ssh lib grep FlexibleBeamModelica_N_2 /var/www/branches/history/master/*.html
/var/www/branches/history/master/2018-05-14 11:14:52..2018-05-15 17:18:02.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/master/2018-05-14 11:14:52..2018-05-15 17:18:02.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/master/2021-04-18 12:44:51..2021-04-19 23:53:17.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="warning">Verify &rarr; Simulate</td></tr>
/var/www/branches/history/master/2021-04-18 12:44:51..2021-04-19 23:53:17.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="warning">Verify &rarr; Simulate</td></tr>
/var/www/branches/history/master/2021-04-19 23:53:17..2021-04-21 00:08:49.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/master/2021-04-19 23:53:17..2021-04-21 00:08:49.html:<tr><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/master/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="better">Simulate &rarr; Verify</td></tr>
casella commented 2 years ago

Thanks @sjoelund! grep does miracles (if you have access to the file system, that is...)

casella commented 2 years ago

@sjoelund, we need you to grep newInst-daemode, though 😄

perost commented 2 years ago

It seems like it started working in this report where I made the NF expand function arguments, and then broke in the next report when I disabled it again because it caused issues (and is not something we generally want to do anyway). That was back in 2018, I couldn't find any report that they've worked since then.

sjoelund commented 2 years ago

@sjoelund, we need you to grep newInst-daemode, though

$ ssh lib grep FlexibleBeamModelica_N_2 /var/www/branches/history/newInst-daemode/*.html
/var/www/branches/history/newInst-daemode/2018-05-05 14:52:54..2018-05-07 14:34:49.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td>ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</td><td class="better">Failed &rarr; FrontEnd</td></tr>
/var/www/branches/history/newInst-daemode/2018-05-05 14:52:54..2018-05-07 14:34:49.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td>ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</td><td class="better">Failed &rarr; FrontEnd</td></tr>
/var/www/branches/history/newInst-daemode/2018-05-07 14:34:49..2018-05-09 14:25:48.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td>ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</td><td class="warning">FrontEnd &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2018-05-07 14:34:49..2018-05-09 14:25:48.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td>ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</td><td class="warning">FrontEnd &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2018-05-09 14:25:48..2018-05-11 14:32:28.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; FrontEnd</td></tr>
/var/www/branches/history/newInst-daemode/2018-05-09 14:25:48..2018-05-11 14:32:28.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; FrontEnd</td></tr>
/var/www/branches/history/newInst-daemode/2018-06-19 22:01:30..2018-06-21 16:04:04.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">FrontEnd &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2018-06-19 22:01:30..2018-06-21 16:04:04.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">FrontEnd &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2018-07-29 01:19:23..2018-08-03 08:53:17.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Templates</td></tr>
/var/www/branches/history/newInst-daemode/2018-07-29 01:19:23..2018-08-03 08:53:17.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Templates</td></tr>
/var/www/branches/history/newInst-daemode/2018-09-25 15:20:33..2018-09-27 08:12:45.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Templates &rarr; Simulate</td></tr>
/var/www/branches/history/newInst-daemode/2018-09-25 15:20:33..2018-09-27 08:12:45.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Templates &rarr; Simulate</td></tr>
/var/www/branches/history/newInst-daemode/2018-10-19 07:50:59..2018-10-21 07:53:56.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/newInst-daemode/2018-10-19 07:50:59..2018-10-21 07:53:56.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/newInst-daemode/2018-10-21 07:53:56..2018-10-25 07:29:36.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
/var/www/branches/history/newInst-daemode/2018-10-21 07:53:56..2018-10-25 07:29:36.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
/var/www/branches/history/newInst-daemode/2018-12-11 15:15:54..2018-12-15 05:28:53.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Compile &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2018-12-11 15:15:54..2018-12-15 05:28:53.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Compile &rarr; Failed</td></tr>
/var/www/branches/history/newInst-daemode/2019-01-22 09:48:04..2019-01-23 13:01:10.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Compile</td></tr>
/var/www/branches/history/newInst-daemode/2019-01-22 09:48:04..2019-01-23 13:01:10.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Compile</td></tr>
/var/www/branches/history/newInst-daemode/2019-06-27 03:39:08..2019-06-29 15:19:48.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="warningPerformance">Compile performance 11.40 &rarr; 21.23</td></tr>
/var/www/branches/history/newInst-daemode/2019-06-29 15:19:48..2019-07-17 04:19:16.html:<tr><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a> <a href="http://libraries.openmodelica.org/branches/newInst-daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.sim">(sim)</a></td><td class="betterPerformance">Compile performance 21.23 &rarr; 3.87</td></tr>
sjoelund commented 2 years ago

@sjoelund, we need you to grep newInst-daemode, though

You can also search your mailbox :)

perost commented 2 years ago

@sjoelund, we need you to grep newInst-daemode, though

You can also search your mailbox :)

I just wget:et the whole index instead :P

AnHeuermann commented 2 years ago

Not sure what we are missing. We set up KLU as linear solver and the initialization is done successfully and it fails at the first time step during the setup with SUNLinSolSetup.

As far as I understand, normally "solver setup failed" really means "singular model". Possibly because of some error in computing the sparse Jacobian, e.g. missing some elements of it.

Could be. I looked into the sparse Jacobian and it had some values, but I didn't check if they made sense. It's a 47x47 sparse Jacobian with 207 non-zero elements. Not sure how the Jacobian should look like for this example.

@peros What did https://github.com/OpenModelica/OMCompiler/commit/75c4b3eca change? Expand arrays given to functions?

perost commented 2 years ago

@Peros What did OpenModelica/OMCompiler@75c4b3e change? Expand arrays given to functions?

Yes, a function call like:

model M
  Real x[3];
equation
  f(x);
end M;

will normally flatten to f(x) with the NF, but with -d=nfExpandFuncArgs or the OF it flattens to f({x[1], x[2], x[3]}).

casella commented 2 years ago

You can also search your mailbox :)

That's 40 GB and growing 😭

casella commented 2 years ago

@sjoelund, I checked your last grep output, but it seems that as we go back in time, the newInst module was really not up to the task of handling complicated multibody models.

Could you also please grep me /var/www/branches/history/daemode/*.html?

I am sure that those examples worked once upon a time, because we published results in this paper in 2017.

AnHeuermann commented 2 years ago

We could also do a git bisection to find the commit which broke this. With a fast PC (to recompile omc and run the model) it should be possible to find it in about 30 minutes. @casella can you tell me a version of OMC where this did work?

sjoelund commented 2 years ago 
$ ssh lib grep FlexibleBeamModelica_N_2 /var/www/branches/history/daemode/*.html        
/var/www/branches/history/daemode/2018-05-11 11:11:15..2018-05-17 20:40:59.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2018-05-11 11:11:15..2018-05-17 20:40:59.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Simulate &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2018-06-19 18:23:55..2018-06-21 15:44:09.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Failed</td></tr>
/var/www/branches/history/daemode/2018-06-19 18:23:55..2018-06-21 15:44:09.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Failed</td></tr>
/var/www/branches/history/daemode/2018-06-21 15:44:09..2018-06-27 11:07:07.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2018-06-21 15:44:09..2018-06-27 11:07:07.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Failed &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2018-10-21 04:46:15..2018-10-25 04:22:18.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
/var/www/branches/history/daemode/2018-10-21 04:46:15..2018-10-25 04:22:18.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
/var/www/branches/history/daemode/2018-12-11 11:31:21..2018-12-15 01:09:03.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Compile &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2018-12-11 11:31:21..2018-12-15 01:09:03.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="better">Compile &rarr; Verify</td></tr>
/var/www/branches/history/daemode/2019-01-22 05:22:24..2019-01-23 09:03:54.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/ScalableTestSuite.html">ScalableTestSuite</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite/files/ScalableTestSuite_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
/var/www/branches/history/daemode/2019-01-22 05:22:24..2019-01-23 09:03:54.html:<tr><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/ScalableTestSuite_noopt.html">ScalableTestSuite_noopt</a></td><td><a href="http://libraries.openmodelica.org/branches/daemode/ScalableTestSuite_noopt/files/ScalableTestSuite_noopt_ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2.err">ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2</a></td><td class="warning">Verify &rarr; Compile</td></tr>
mahge commented 2 years ago

Here is a simple python script that can scan the regression reports. We can use this when we do not have access to the actual filesystem.


import requests
# import base64
import argparse
from bs4 import BeautifulSoup
from termcolor import colored
from tqdm import tqdm

import os
# os.system('color')

def print_green(string_data, **kwargs):
    print(colored(string_data, "green"), **kwargs, flush=True)

def scan_reg_report_for_model(reg_url, in_model_name):
    response = requests.get(reg_url)

    # Sometimes we have urls with missing pages.
    if response.status_code != 200:
        print ("Skipping invalid url " + reg_url + ": " + str(response.status_code))
        return

    soup = BeautifulSoup(response.text, 'html.parser')

    tables = soup.find_all("table")
    models_reg_table = tables[3]

    model_reg_links = models_reg_table.find_all("a", href=True)

    for model_reg_link in model_reg_links:
        model_name = model_reg_link.get_text(strip=True)

        if (model_name == in_model_name):
            print_green(reg_url)
            break

def find_model_regressions():

    parser = argparse.ArgumentParser()
    parser.add_argument("--model",
                        required=True,
                        help='The model name to search regressions for. This should be the fully qualifed name.')
    parser.add_argument("--branch",
                        choices=['daemode', 'master-fmi', 'master', 'newInst-daemode', 'newInst', 'oldInst'],
                        default='master',
                        help='The branch name to search regressions in. This should be the fully qualifed name.')
    parser.add_argument("--order",
                        choices=['latest', 'oldest'],
                        default='latest',
                        help='Start scanning from the LATEST or the OLDEST reports')
    args = parser.parse_args()

    model_name = args.model
    branch_name = args.branch
    scan_order = args.order

    print_green("Scanning '" + branch_name + "' for model '" + model_name + "'")

    response = requests.get("https://libraries.openmodelica.org/branches/history/" + branch_name + "/00_history.html")
    soup = BeautifulSoup(response.text, 'html.parser')

    regg_links_s = soup.find_all("a", href=True)

    print_green("Total number of reports to scan: " + str(len(regg_links_s)))

    if scan_order == "latest":
        regg_links_s = list(reversed(regg_links_s))

    for regg_link_s in tqdm(regg_links_s, desc = 'Scanning: '):
        report_link = regg_link_s['href']
        # print(report_link)
        scan_reg_report_for_model(report_link, model_name)

# scan_reg_report_for_model("https://libraries.openmodelica.org/branches/history/daemode/2020-11-13%2001:21:11..2020-11-15%2012:07:40.html","ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2")

find_model_regressions()

Save this code as find_regs.py or something and you can call it as follows:

# Install some required modules
pip3 install tqdm termcolor BeautifulSoup4
# Run script
python3 find_regs.py --model=ScalableTestSuite.Mechanical.FlexibleBeam.ScaledExperiments.FlexibleBeamModelica_N_2 --branch=daemode --order=oldest

You need to specify the branch name (defaults to master if missing) and the qualified model name (required always) you want to search for. You can also specify the order in which to scan reports, --order=oldest OR --order=latest

Note that this will search for the specified model name exactly and can give you false positives if the model name exists in two different libraries, e.g. the above model exists in both ScalableTestSuite and ScalableTestSuite_noopt libraries.

Reports for daemode

http://libraries.openmodelica.org/branches/history/daemode/2018-05-11%2011:11:15..2018-05-17%2020:40:59.html http://libraries.openmodelica.org/branches/history/daemode/2018-06-19%2018:23:55..2018-06-21%2015:44:09.html http://libraries.openmodelica.org/branches/history/daemode/2018-06-21%2015:44:09..2018-06-27%2011:07:07.html http://libraries.openmodelica.org/branches/history/daemode/2018-10-21%2004:46:15..2018-10-25%2004:22:18.html http://libraries.openmodelica.org/branches/history/daemode/2018-12-11%2011:31:21..2018-12-15%2001:09:03.html http://libraries.openmodelica.org/branches/history/daemode/2019-01-22%2005:22:24..2019-01-23%2009:03:54.html

Reports for newInst-daemode http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-05-09%2014:25:48..2018-05-11%2014:32:28.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-06-19%2022:01:30..2018-06-21%2016:04:04.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-07-29%2001:19:23..2018-08-03%2008:53:17.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-09-25%2015:20:33..2018-09-27%2008:12:45.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-10-19%2007:50:59..2018-10-21%2007:53:56.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-10-21%2007:53:56..2018-10-25%2007:29:36.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2018-12-11%2015:15:54..2018-12-15%2005:28:53.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2019-01-22%2009:48:04..2019-01-23%2013:01:10.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2019-06-27%2003:39:08..2019-06-29%2015:19:48.html http://libraries.openmodelica.org/branches/history/newInst-daemode/2019-06-29%2015:19:48..2019-07-17%2004:19:16.html

mahge commented 2 years ago

I think the model progressed from Simulate -> Verify -> Compile in this two reports

Simulate -> Verify https://libraries.openmodelica.org/branches/history/newInst-daemode/2018-10-19%2007:50:59..2018-10-21%2007:53:56.html Verify -> Compile https://libraries.openmodelica.org/branches/history/newInst-daemode/2018-10-21%2007:53:56..2018-10-25%2007:29:36.html

So the culprit might be one of the commits in https://libraries.openmodelica.org/branches/history/newInst-daemode/2018-10-21%2007:53:56..2018-10-25%2007:29:36.html

mahge commented 2 years ago

@casella Can this be the cause?

https://github.com/casella/ScalableTestSuite/commit/18838b56973d9bd293f2adcc7b1f1ff6754b7ab9

It is part of of the changes in the offending regression report.

casella commented 2 years ago

@casella Can this be the cause?

Thanks @mahge, you nailed it 😄

@AnHeuermann, I don't remember why I changed that precision in the library three years ago. But at least now we have a nice repeatable test case. BTW, also the other cases were more or less randomly succeeding or failing, so I guess these models are borderline in terms of numerical robustness.

Anyway, if you run the FlexibleBeamModelica_N_2 example with tol = 1e-6, it works fine. With tol = 1e-7 and -lv=LOG_SOLVER, the simulation goes on for a few steps

...
call solver from 0.00098049 to 0.0010005 (stepSize: 2.00100050025012e-05)
##IDA## new step from 0.000980490245122561 to 0.00100050025012506
##IDA## Finished Integrator step.
finished solver step 0.001
##IDA## do event update at 0.00100000001001002

and then fails 

Simulation process failed. Exited with code 0xffffffffffffffff.
#### IDA error message #####
 -> error code -6
 -> module IDAS
 -> function IDACalcIC
 Message: The linear solver setup failed unrecoverably.
##IDA## IDACalcIC run status -6.
Iterations : 0

##IDA## first event iteration failed. Start next try without line search!
#### IDA error message #####
##IDA## IDACalcIC run status -6.
Iterations : 0

##IDA## discrete update failed flag -6!

I'm not exactly sure what happens here, but there is an event at time = 0.002, because of the conditional equation that computes the force variable. BTW, this is time event, that we still handle as as state event. If the zero-crossing function were f_zc = time - 0.002, this should be totally straightforward to solve. Recall the discussion we had on event handling for spatialDistribution.

Apparently something goes wrong here, I guess it shoud be investigated.

casella commented 2 years ago

@phannebohm I understand you are working on event handling now, we promised RTE to fix this issue in the last DFD, would you mind having a look? The ticket is very long, but the relevant comments are the last few ones.

Thanks!

casella commented 1 year ago

@phannebohm please consider this while working on daeMode event handling.