NREL-Sienna / InfrastructureSystems.jl

Utility package for Sienna's simulation infrastructure
https://nrel-sienna.github.io/InfrastructureSystems.jl/
BSD 3-Clause "New" or "Revised" License
39 stars 20 forks source link

Catch `ResolverError` to reduce possibility misleading failures from integration tests #223

Closed nickrobinson251 closed 3 years ago

nickrobinson251 commented 3 years ago

When looking into #222, i noticed this package had integration tests with downstream packages - which is great. And they even caught the breaking change in #220 (which were then fixed in #221)

But i noticed one tiny way that might be improved: the integration tests currently allow false negatives, which slightly reduces the signal-noise ratio. This PR changes it so that a PR doesn't fail the integration test if it introduces a breaking change but also bumps to a breaking version; it changes the the Julia code that runs the test to catch a ResolverError and exit with success in that case.

(This is not related to #220, this is just something i saw in passing when investigating if we could further reduce the risk of accidental breaking changes, which hopefully this does by meaning we have even more reason to trust the integration tests)

codecov[bot] commented 3 years ago

Codecov Report

Merging #223 (a190166) into master (97ebf80) will decrease coverage by 0.04%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #223      +/-   ##
==========================================
- Coverage   75.00%   74.95%   -0.05%     
==========================================
  Files          43       43              
  Lines        3052     3051       -1     
==========================================
- Hits         2289     2287       -2     
- Misses        763      764       +1     
Flag Coverage Δ
unittests 74.95% <ø> (-0.05%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/utils/utils.jl 60.33% <0.00%> (-0.43%) :arrow_down:
src/utils/lazy_dict_from_iterator.jl 96.15% <0.00%> (-0.15%) :arrow_down:
jd-lara commented 3 years ago

@nickrobinson251 thanks. I will implement this in PSY too.