Climatology test doesn't work with null values if zspan isn't provided

[lgarzio]

I'm having an issue with the climatology_test not handling nans if zspan isn't provided. I'm using the latest version of ioos_qc (2.0.1), and according to the ClimatologyConfig documentation, zspan is optional.

from ioos_qc.qartod import ClimatologyConfig, climatology_test
c = ClimatologyConfig()
c.add(tspan=['2021-06-01', '2021-09-01'], vspan=[3.4, 5])

inp = np.array([0.     ,     np.nan, 0.     ,     np.nan,     np.nan,     np.nan,     4.16743, 4.23101,     4.23322])

t = np.array(['2021-07-16T19:01:01.313999872', '2021-07-16T19:01:02.315000064',
       '2021-07-16T19:01:02.903000064', '2021-07-16T19:01:03.903000064',
       '2021-07-16T19:01:04.903000064', '2021-07-16T19:01:05.903000064',
       '2021-07-16T19:01:06.903000064', '2021-07-16T19:01:07.336000000',
       '2021-07-16T19:01:07.903000064'], dtype='datetime64[ns]')

z = np.array([0.     ,     np.nan, 0.     ,     np.nan,     np.nan,     np.nan,     0.08931513, 0.15878244,     0.11908684])

result = climatology_test(config=c,
                        inp=inp,
                        tinp=t,
                        zinp=z)

result
masked_array(data=[2, 1, 2, 1, 1, 1, 2, 2, 2],
             mask=False,
       fill_value=999999,
            dtype=uint8)

This result is incorrect, however if I add zspan to the configuration, it works correctly:

c = ClimatologyConfig()
c.add(tspan=['2021-06-01', '2021-09-01'],
      vspan=[3.4, 5],
      zspan=[0, 1000])

result = climatology_test(config=c,
                        inp=inp,
                        tinp=t,
                        zinp=z)
result
masked_array(data=[3, 9, 3, 9, 9, 9, 1, 1, 1],
             mask=False,
       fill_value=999999,
            dtype=uint8)

[kwilcox]

Thanks, this is a bug! There was a fix proposed in https://github.com/ioos/ioos_qc/pull/35 but I never extracted the change out from all of the conversion to py2 stuff. I can look at doing so now that you have provided a nice test case.

[eldobbins]

I am also having trouble with zspan and climatology. In my case, the test isn't working at all if I don't specify zspan. Here's my test code:

cc = qartod.ClimatologyConfig()
cc.add(
    vspan=(15, 16),  # range of valid values
    tspan=("2022-01-01", "2022-02-01"),
    zspan=(-100,100)
)

qc_config = {
    "climatology_test": {
        "config": cc
    }
}

qc = QcConfig({'qartod': qc_config})
qc.run( inp=[20], tinp=[pd.to_datetime('2022-01-15')], zinp=[10])

With the zspan line in the config, the test evaluates (and returns 3). If I comment out the zspan line, the test does not evaluate (it returns 2).

[iwensu0313]

I am also having trouble with zspan and climatology. In my case, the test isn't working at all if I don't specify zspan. Here's my test code:

cc = qartod.ClimatologyConfig()
cc.add(
    vspan=(15, 16),  # range of valid values
    tspan=("2022-01-01", "2022-02-01"),
    zspan=(-100,100)
)

qc_config = {
    "climatology_test": {
        "config": cc
    }
}

qc = QcConfig({'qartod': qc_config})
qc.run( inp=[20], tinp=[pd.to_datetime('2022-01-15')], zinp=[10])

With the zspan line in the config, the test evaluates (and returns 3). If I comment out the zspan line, the test does not evaluate (it returns 2).

Ran this example test code with the changes in this PR https://github.com/ioos/ioos_qc/pull/104 and it works now. A value of 3 is returned whether you specify zspan or not

[iwensu0313]

Still looking into fixing the behavior seen in the original test code.