BuildingsPy (or Travis) produces inconsistent results

[Mathadon]

Sometimes not all unit tests are executed. E.g. in https://travis-ci.org/open-ideas/IDEAS/jobs/383240293 not all tests are executed, while they are executed correctly in https://travis-ci.org/open-ideas/IDEAS/jobs/382014283.

In the former case the travis log contains:

export TEST_ARG="make test-dymola    PACKAGE=IDEAS.Fluid.{HeatExchangers,HeatPumps,Interfaces,MassExchangers,MixingVolumes,Movers,Sensors,Sources,Storage}"
...
(cd ../../../../IDEAS && python ../bin/runUnitTests.py --batch --single-package IDEAS.Fluid.Storage --tool dymola)

instead of

export TEST_ARG="make test-dymola    PACKAGE=IDEAS.Fluid.{HeatExchangers,HeatPumps,Interfaces,MassExchangers,MixingVolumes,Movers,Sensors,Sources,Storage}"
...
(cd ../../../../IDEAS && python ../bin/runUnitTests.py --batch --single-package IDEAS.Fluid.{HeatExchangers,HeatPumps,Interfaces,MassExchangers,MixingVolumes,Movers,Sensors,Sources,Storage} --tool dymola)

I.e. in the former case only the tests for IDEAS.Fluid.Storage are run. Other tests in the same series (https://travis-ci.org/open-ideas/IDEAS/builds/383240305?utm_source=github_status&utm_medium=notification) have the same problem: only the last set of unit tests (e.g. FixedResistances or Storage or Utilities) within the same set (within the curly brackets) is executed.

This is using the master branch of BuildingsPy. Have you had similar issues before? It looks a bit like black magic to me..

Thanks!

[mwetter]

@Mathadon : I have not seen this behavior. Can you reproduce it? Note also the telephone icon on line 384 on https://travis-ci.org/open-ideas/IDEAS/jobs/383240293 Does this mean call IDEAS support?

[Mathadon]

@mwetter it seems that I can, I restarted the IBPSA master build: https://travis-ci.org/ibpsa/modelica-ibpsa/jobs/382206551 . Same problem here. This makes me think that something is going on at travis..?

Sure not, there exists no such thing! I don't see the icon btw :)

[mwetter]

Indeed. I verified the Makefile and cannot explain this behavior. Do you want to send an email to support@travis-ci.com?

[Mathadon]

@mwetter I think I found it. I'm not sure whether you know about brace expansion in shell scripts? See https://stackoverflow.com/questions/19291746/escape-curly-braces-in-unix-shell-script .

When I go to IBPSA/Resources/Scripts/travis in my VM and execute make test-dymola PACKAGE=IDEAS.{Media,Utilities} then this also results 'incorrectly' in:

(cd ../../../../IDEAS && python ../bin/runUnitTests.py --batch --single-package IDEAS.Utilities --tool dymola)
Regression tests are only run for the following package:
  IDEAS.Utilities

you see what happens if you do

echo make test-dymola    PACKAGE=IDEAS.{Media,Utilities}
>>> make test-dymola PACKAGE=IDEAS.Media PACKAGE=IDEAS.Utilities

what does work is (added quotes):

make test-dymola    PACKAGE="IDEAS.{Media,Utilities}"
>>> (cd ../../../../IDEAS && python ../bin/runUnitTests.py --batch --single-package IDEAS.{Media,Utilities} --tool dymola)
>>> Regression tests are only run for the following packages:
>>>   IDEAS.Media
>>>   IDEAS.Utilities

My best guess is that travis updated some software that now interprets our code differently.. But I think that the 'bug' is on our side of the code. I'll add the quotes to verify.

edit: results pending here : https://travis-ci.org/ibpsa/modelica-ibpsa/builds/383343540?utm_source=github_status&utm_medium=notification

[mwetter]

@Mathadon I think this is what is happening. Please make a pull request if quotes fix the problem

[Mathadon]

now I'm also seeing the telephone thing.. (which doesn't look like a telephone here on github except if you edit the message)

*** Warning: CavityWalls.mat: outerWall.layMul.monLay[2].port_a.T has absolute and relative error = 6.934e-02, 2.538e-04

not sure what that is about