NREL / OpenStudio-measure-tester-gem

Other
3 stars 0 forks source link

Can we set a timeout for a measure test, maybe 30 or 60 minutes? #13

Open DavidGoldwasser opened 6 years ago

DavidGoldwasser commented 6 years ago

1 test out of 22 gets stuck in a branch we are working on. I don't want to comment out the test, because it is identifying an issue, but I can't run the rest of the dashboard because of this one. https://github.com/NREL/OpenStudio-measures/blob/feature/update_os_stds_methods/nrel_dev/create_bar_from_building_type_ratios/tests/create_bar_from_building_type_ratios_test.rb#L258-L268

DavidGoldwasser commented 6 years ago

@macumber with 2.6.0 on mac I still get stuck here. Did you not get stuck on this in Windows? I can just turn of this test for this case, but it seems like a timeout would be good.

Link above isn't valid anymore. here is an updated link https://github.com/NREL/OpenStudio-measures/blob/260_testing/nrel_dev/create_bar_from_building_type_ratios/tests/create_bar_from_building_type_ratios_test.rb#L258-L270

macumber commented 6 years ago

@DavidGoldwasser can you provide the exact command you are running and your environment?

DavidGoldwasser commented 6 years ago

I confirmed that this same test hangs when I run the measure test with openstudio my_test.rbcommand outside of the tester framework. So this issue is mainly about adding a timeout. I could probably add an OpenStudio issue on why it hangs. This is 1 out of 22 tests, it does have some comments about previous inconsistent failures.

this test is failing intermittently due to unexpected warning Office WholeBuilding - Md Office doesn't have the expected floor area (actual 41,419 ft^2, target 40,000 ft^2) 40,709, 40,000 footprint size is always fine, intersect is probably creating issue with extra surfaces on top of each other adding the extra area haven't seen this on other partial story models Error: Surface 138 This planar surface shares the same SketchUp face as Surface 143. This error cannot be automatically fixed. The surface will not be drawn.

     Error:  Surface 91
 This planar surface shares the same SketchUp face as Surface 141.
     This error cannot be automatically fixed.  The surface will not be drawn.

     Error:  Surface 125
 This planar surface shares the same SketchUp face as Surface 143.
     This error cannot be automatically fixed.  The surface will not be drawn.
DavidGoldwasser commented 6 years ago

@macumber Ok, while this command locks up

openstudio create_typical_building_from_model_test.rb

This command finishes fine

ruby create_typical_building_from_model_test.rb

in addition to the second command not locking up, the specific test in question in the suite passes the test.

DavidGoldwasser commented 6 years ago

Using the tester gem locks up if the command below is run from within the individual measure in question, and also when it is run from a parent directory with many measures.

rake openstudio:test

macumber commented 6 years ago

Those two commands in your post 2 up look the same?

DavidGoldwasser commented 6 years ago

Those two commands in your post 2 up look the same?

@macumber The first one that locks up calls openstudio the other calls ruby

macumber commented 6 years ago

Can you try: openstudio --verbose create_typical_building_from_model_test.rb