search

modelica / ModelicaStandardLibrary

Free (standard conforming) library to model mechanical (1D/3D), electrical (analog, digital, machines), magnetic, thermal, fluid, control systems and hierarchical state machines. Also numerical functions and functions for strings, files and streams are included.
https://doc.modelica.org
BSD 3-Clause "New" or "Revised" License
452 stars 165 forks source link

MSL 4.1.0 Regressions - Blocks #4296

Open GallLeo opened 5 months ago

GallLeo commented 5 months ago

The following models fail in result comparison. Status: 7ac79067 (2024-01-30)

beutlich commented 5 months ago

@HansOlsson Is this due to #4151?

@GallLeo Please add the appropriate labels for library and version.

HansOlsson commented 5 months ago

@HansOlsson Is this due to #4151?

@GallLeo Please add the appropriate labels for library and version.

I'm a bit lost. What is the error? Clicking on the links just give a bunch of green texts.

beutlich commented 5 months ago

Try https://www.ltx.de/download/MA/Compare_MSL_v4.1.0/Compare/Modelica/20240129232335/Modelica.Blocks.Examples.Noise.NormalNoiseProperties/CSVCompare/NormalNoiseProperties_report.html where the mean output error is the reason for result failure.

HansOlsson commented 5 months ago

Try https://www.ltx.de/download/MA/Compare_MSL_v4.1.0/Compare/Modelica/20240129232335/Modelica.Blocks.Examples.Noise.NormalNoiseProperties/CSVCompare/NormalNoiseProperties_report.html where the mean output error is the reason for result failure.

Ah, yes, the protected variable mu is now completely different due to PR #4151 - but y should be the same; so just reference update (or removing from comparison).

beutlich commented 5 months ago

Yes, removing from comparison might be good approach since protected anyway.

GallLeo commented 5 months ago

@GallLeo Please add the appropriate labels for library and version.

I'm a bit lost. What is the error? Clicking on the links just give a bunch of green texts.

Sorry, I removed the automatic browser links. They linked to the test case, not to the comparison report. Only the comparison links count.

HansOlsson commented 5 months ago

@GallLeo Please add the appropriate labels for library and version.

I'm a bit lost. What is the error? Clicking on the links just give a bunch of green texts.

Sorry, I removed the automatic browser links. They linked to the test case, not to the comparison report. Only the comparison links count.

Note that the first two seems to be reference updates, unrelated to #4151

HansOlsson commented 5 months ago

Possibly different random sequence for the first two.

beutlich commented 5 months ago

Possibly different random sequence for the first two.

This needs some more evidence as I am not aware on RNG changes.

beutlich commented 5 months ago

I run git bisect on Modelica.Blocks.Examples.Noise.ActuatorWithNoise and got:

a2cce87f7ab1a2a59f5263867c507e31241c23fd is the first bad commit

This points to #3727/#3728 and even @maltelenz' comment https://github.com/modelica/ModelicaStandardLibrary/issues/3727#issuecomment-763449287 where it is stated that the MSL 4.0.0 reference result is invalid.

HansOlsson commented 1 month ago

This was discussed at the latest MAP-Lib meeting (2024-05-14), but the ones that I should update already have a PR linked to this one - it is just waiting to be reviewed and merged.

beutlich commented 1 month ago

This was discussed at the latest MAP-Lib meeting (2024-05-14),

I would help if the outcome of this kind of discussion somehow gets back to the originating issues/PRs.

casella commented 3 weeks ago

@GallLeo everything should be ready for the new results generation