search

modelica / ModelicaStandardLibrary

Free (standard conforming) library to model mechanical (1D/3D), electrical (analog, digital, machines), magnetic, thermal, fluid, control systems and hierarchical state machines. Also numerical functions and functions for strings, files and streams are included.
https://doc.modelica.org
BSD 3-Clause "New" or "Revised" License
479 stars 169 forks source link

Missing reference results #2968

Closed maltelenz closed 4 years ago

maltelenz commented 5 years ago

All of the below concerns version 3.2.3.

These models do not seem to have any reference results in the reference results repository (or traces that one was attempted). They all have experiment annotations though:

These models seem to be missing the csv file:

christiankral commented 5 years ago

@GallLeo Can you more or less create the test results automatically by reading the model experiment annotation in an automated way, if I provide the proper list of variables in a file named comparisonSignals.txt?

christiankral commented 5 years ago

Refs https://groups.google.com/forum/#!msg/map-lib/jDbvOWGbQNY/klMtdIgHAgAJ

beutlich commented 4 years ago

Will be resolved by #3348.

maltelenz commented 4 years ago

Will #3348 also be merged into the 3.2.3 branch? If it is only for master/4.0.0, I don't agree that it resolves this issue, which is for 3.2.3.

beutlich commented 4 years ago

Will #3348 also be merged into the 3.2.3 branch? If it is only for master/4.0.0, I don't agree that it resolves this issue, which is for 3.2.3.

I do not think that anyone is going to repeat the MSL v3.2.3 regression run. My focus is MSL v4.0.0 now. Since #3348 is pretty time-consuming (with all the new models, renamed models, renamed signals or even non-transient simulation results (see e.g. #3349)) and since this issue it is not a real bug of MSL v3.2.3 (just a shortcoming of the previous regression runs), it will be good enough if we fix it for MSL v4.0.0.

maltelenz commented 4 years ago

Missing the reference results means no tools can verify they correctly handle these models.

Focusing on 4.0.0 is certainly a valid way to prioritize, but it still doesn't fix this problem for 3.2.3, and therefore doesn't fix this issue.

beutlich commented 4 years ago

This is a process issue since previously the introduced new example models were somehow forgotten. For example, I reminded @GallLeo to update the correspondig reference signals in case of new models (see https://github.com/modelica/ModelicaStandardLibrary/pull/2738#issuecomment-429625984 for an example) but it never happened. With #3348 all reference signals are part of the repository and also in responsibility of the library officers. I do hope that this fixes the process in general, whereas it does not fix the MSL v3.2.3 shortcoming you reported. Sorry for that.

beutlich commented 4 years ago

Meanwhile, CSV files for

are available by https://github.com/modelica/MAP-LIB_ReferenceResults/commit/aa90b8158c02c8f2d5ad6e12ddc05a329aace596.

beutlich commented 4 years ago

Should all be fixed by the newly created reference results of upcoming MSL 3.2.3 Build.4: https://github.com/modelica/MAP-LIB_ReferenceResults/tree/v3.2.3+build.4