search

lbl-srg / BuildingsPy

Python modules for automating Modelica simulations and for running unit test for the Buildings library
82 stars 46 forks source link

Missing reference results does not trigger an error in batch mode #531

Closed mwetter closed 10 months ago

mwetter commented 10 months ago

If CI tests are run in batch mode, then a missing reference result does not trigger an error.

To reproduced, checkout out IBPSA library, commit dd1aace714c54f74603cfa505529738f943109a2

Run

$ ../bin/runUnitTests.py -s IBPSA.Electrical.DC.Conversion.Examples -b
Regression tests are only run for the following package:
  IBPSA.Electrical.DC.Conversion.Examples
Number of models   : 1004
          blocks   : 128
          functions: 133
Using 1 of 48 processors to run unit tests for dymola.
Generated 1 regression tests.

Comparison files output by funnel are stored in the directory 'funnel_comp' of size 0.0 MB.
Run 'python -c "import buildingspy.development.regressiontest as t; t.Tester(tool=\"dymola\").report()"'
to access a summary of the comparison results.

Script that runs unit tests had 0 warnings and 0 errors.

See 'simulator-dymola.log' for details.
Unit tests completed successfully.

Execution time = 5.998 s

If run without the -b flag, the plot window with the new reference results opens.