There is a conditional in adjust_prs_val_units() that checks if prs_val0: to perform unit adjustment on the prs axis. This is a silent bug in this conditional where unit adjustment is not being performed when prs_val0 == 0 because 0 == False in boolean form.
There are several derived variables that define prs_low0=0. For example, notice how CLDLOW_TAU_1.3_9.4_MISR has a prs_low0=0 below. For the dataset I was working with, this value is never used due to the silent bug. No prs axis adjustments occur and the prs low value from the dataset (-0.001) is used which results in incorrect results.
What did you expect to happen? Are there are possible answers you came across?
The conditional should be updated to if prs_val0 is not None: since prs_val0 is optional. This logic is implemented correctly on the dev branch, cdat-migration-fy24.
def adjust_prs_val_units(
prs: "FileAxis", prs_val: float, prs_val0: Optional[float]
) -> float:
"""Adjust the prs_val units based on the prs.id"""
# COSP v2 cosp_pr in units Pa instead of hPa as in v1
# COSP v2 cosp_htmisr in units m instead of km as in v1
adjust_ids = {"cosp_prs": 100, "cosp_htmisr": 1000}
if prs_val0 is not None:
prs_val = prs_val0
if prs.id in adjust_ids.keys() and max(prs.getData()) > 1000:
prs_val = prs_val * adjust_ids[prs.id]
return prs_val
Minimal Complete Verifiable Example (MVCE)
No response
Relevant log output
No response
Anything else we need to know?
How I found it
In PR #794, I found the following variables had large diffs. I stepped through the code on main and the dev branch and found that main doesn't perform prs unit adjustment while the dev branch does.
These are the kinds of issues that can be easy to miss, which is why linters and type checkers (e.g., flake8 and mypy) are valuable and great to use as preventative tools.
What happened?
Overview
There is a conditional in
adjust_prs_val_units()
that checksif prs_val0:
to perform unit adjustment on the prs axis. This is a silent bug in this conditional where unit adjustment is not being performed whenprs_val0 == 0
because0 == False
in boolean form.Related code: https://github.com/E3SM-Project/e3sm_diags/blob/43f5f74057929c4bedddfab58be63f838ee36abe/e3sm_diags/derivations/acme.py#L433-L446
Example Case
There are several derived variables that define
prs_low0=0
. For example, notice howCLDLOW_TAU_1.3_9.4_MISR
has aprs_low0=0
below. For the dataset I was working with, this value is never used due to the silent bug. No prs axis adjustments occur and the prs low value from the dataset (-0.001) is used which results in incorrect results.https://github.com/E3SM-Project/e3sm_diags/blob/43f5f74057929c4bedddfab58be63f838ee36abe/e3sm_diags/derivations/acme.py#L1378-L1393
What did you expect to happen? Are there are possible answers you came across?
The conditional should be updated to
if prs_val0 is not None:
sinceprs_val0
is optional. This logic is implemented correctly on the dev branch,cdat-migration-fy24
.Minimal Complete Verifiable Example (MVCE)
No response
Relevant log output
No response
Anything else we need to know?
How I found it
In PR #794, I found the following variables had large diffs. I stepped through the code on
main
and the dev branch and found thatmain
doesn't perform prs unit adjustment while the dev branch does.Affects the following derived variables:
https://github.com/E3SM-Project/e3sm_diags/blob/43f5f74057929c4bedddfab58be63f838ee36abe/e3sm_diags/derivations/acme.py#L1362-L1409
Environment
main
and all versions ofe3sm_diags
that hascosp_bin_sum()