nasa / prog_models

The NASA Prognostic Model Package is a Python framework focused on defining and building models for prognostics (computation of remaining useful life) of engineering systems, and provides a set of prognostics models for select components developed within this framework, suitable for use in prognostics applications for these components.
123 stars 50 forks source link

Bug fixes in tests for uav model and surrogate #577

Closed kjjarvis closed 1 year ago

kjjarvis commented 1 year ago

Two bugs were found when the change to dt in simulation was made to prevent floating point errors (see changes in hotfix v1.5.1 for more details):

  1. test_uav_model: A loop had incorrect indexing. Issue has been fixed to loop through the appropriate interpolated times vector.
  2. test_surrogates: The floating point change caused some tests to fail due to slightly larger differences when comparing simulated results. The delta values have been increased accordingly
github-actions[bot] commented 1 year ago
Benchmarking Results From: Test Time (s)
import main 0.15207959999999998
import thrown object 0.5615989000000001
model initialization 0.16558209999999995
set noise 0.6982659999999998
simulate 0.5191995
simulate with saving 1.4789527999999996
simulate with saving, dt 1.8954313999999997
simulate with printing results, dt 2.3668811000000005
Plot results 15.250445200000001
Metrics 0.03874179999999683
Surrogate Model Generation 2.2835342999999995
surrogate sim 1.519647599999999
surrogate sim, dt 3.930498100000001
To: Test Time (s)
import main 0.14291579999999993
import thrown object 0.5362501
model initialization 0.16452390000000028
set noise 0.7423823999999999
simulate 0.5120502999999998
simulate with saving 1.4575534999999995
simulate with saving, dt 1.8611713000000005
simulate with printing results, dt 2.323768199999999
Plot results 15.1747536
Metrics 0.038767100000001165
Surrogate Model Generation 2.2703247999999974
surrogate sim 1.5095929999999989
surrogate sim, dt 3.865531100000002
github-actions[bot] commented 1 year ago
Benchmarking Results [Update] From: Test Time (s)
import main 0.14531490000000002
import thrown object 0.5841112000000002
model initialization 0.16548529999999984
set noise 0.7048641
simulate 0.5181423999999999
simulate with saving 1.4566248999999996
simulate with saving, dt 1.8840173
simulate with printing results, dt 2.344428099999999
Plot results 15.2095378
Metrics 0.039116099999997545
Surrogate Model Generation 2.2711982000000006
surrogate sim 1.5027636999999991
surrogate sim, dt 3.849535000000003

To:

Test Time (s)
import main 0.14403220000000005
import thrown object 0.5434844000000001
model initialization 0.16763100000000009
set noise 0.7165444000000001
simulate 0.5115696999999999
simulate with saving 1.4592342999999994
simulate with saving, dt 1.8742109999999998
simulate with printing results, dt 2.330280599999999
Plot results 15.3306907
Metrics 0.03909979999999891
Surrogate Model Generation 2.285427000000002
surrogate sim 1.5082982999999999
surrogate sim, dt 3.8402299999999983
codecov-commenter commented 1 year ago

Codecov Report

:exclamation: No coverage uploaded for pull request base (hotfix/v1.5.1@c61b8fa). Click here to learn what that means. The diff coverage is n/a.

@@               Coverage Diff                @@
##             hotfix/v1.5.1     #577   +/-   ##
================================================
  Coverage                 ?   84.50%           
================================================
  Files                    ?       45           
  Lines                    ?     3504           
  Branches                 ?        0           
================================================
  Hits                     ?     2961           
  Misses                   ?      543           
  Partials                 ?        0