NREL / buildstockbatch

Other
20 stars 13 forks source link

Collect simulation errors in results json #437

Closed nweires closed 3 months ago

nweires commented 4 months ago

Pull Request Description

Copy OpenStudio errors into each task's results json file, so they can more easily be summarized and surfaced after the job is complete. (We are viewing them from a python notebook that shows other summary statistics about a job.)

Example output for one simulation (error caused by a mismatch in package version numbers):

{
  "started_at": "20240108T153906Z",
  "completed_at": "20240108T153921Z",
  "completed_status": "Fail",
  "building_id": 14,
  "step_failures": [
    {
      "measure_dir_name": "HPXMLtoOpenStudio",
      "step_errors": ["undefined method `setTranslateFileWithRelativePath' for #<OpenStudio::Model::ScheduleFile:0x000058fd7f7f1108>\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/resources/schedules.rb:1499:in `create_schedule_file'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/resources/geometry.rb:533:in `apply_occupants'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:567:in `add_num_occupants'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:445:in `create_unit_model'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:182:in `block in run'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:174:in `each'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:174:in `each_with_index'\n/var/simdata/openstudio/resources/hpxml-measures/HPXMLtoOpenStudio/measure.rb:174:in `run'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/util/measure.rb:517:in `apply_measure'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/util/measure.rb:114:in `block in apply_measures'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/util/measure.rb:67:in `each_index'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/util/measure.rb:67:in `apply_measures'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/jobs/run_os_measures.rb:70:in `perform'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/run.rb:291:in `step'\n:/ruby/2.7.0/gems/openstudio-workflow-2.3.1/lib/openstudio/workflow/run.rb:233:in `run'\n:/openstudio_cli.rb:1190:in `execute'\n:/openstudio_cli.rb:820:in `execute'\n:/openstudio_cli.rb:1987:in `<main>'\neval:193:in `eval'\neval:193:in `require_embedded_absolute'\neval:178:in `block in require_embedded'\neval:172:in `each'\neval:172:in `require_embedded'\neval:131:in `require'\neval:3:in `<main>'"]
    }
  ],
  "upgrade": 0
}

Also, unzip the test file results_job1.json.gz to make it human-readable. (It's un-zipped before being used in the test anyway.)

Checklist

Not all may apply

github-actions[bot] commented 4 months ago

File Coverage
All files 87% :white_check_mark:
base.py 92% :white_check_mark:
exc.py 57% :white_check_mark:
hpc.py 78% :white_check_mark:
local.py 70% :white_check_mark:
postprocessing.py 85% :white_check_mark:
utils.py 92% :white_check_mark:
cloud/docker_base.py 88% :white_check_mark:
sampler/base.py 79% :white_check_mark:
sampler/downselect.py 33% :white_check_mark:
sampler/precomputed.py 93% :white_check_mark:
sampler/residential_quota.py 61% :white_check_mark:
test/shared_testing_stuff.py 85% :white_check_mark:
test/test_docker.py 33% :white_check_mark:
test/test_local.py 97% :white_check_mark:
test/test_validation.py 97% :white_check_mark:
workflow_generator/base.py 90% :white_check_mark:
workflow_generator/commercial.py 53% :white_check_mark:
workflow_generator/residential_hpxml.py 86% :white_check_mark:

Minimum allowed coverage is 33%

Generated by :monkey: cobertura-action against f4a05036b658650189acc1a4d393e074bcaf0c14