Closed reedan88 closed 5 years ago
There is some inconsistency across the most of the tests as to how to encode time. The docstring for the rate of change test specifically types tinp as np.datetime64 arrays, but doesn't enforce it, while the flat line test docstring says datetime64 or list of seconds are acceptable.
There are two problems that I see; 1. we need to enforce the type time input or at least explicitly state it in the docstrings. 2. If we are allowing np.datetime64, we need to explicitly say what the units are (nanoseconds vs milliseconds vs seconds).
Since most of the tests make use of at least numpy, as well as pandas, I don't think we can avoid np.datetime64. I've taken a look into the flat_line_test code closely and I think we should replace line 463 in qartod.py:
time_interval = np.median(np.diff(tinp)).astype(float)
with the following two lines:
time_interval = np.median(np.diff(tinp)).astype(np.timedelta64)
time_interval = np.timedelta64(time_interval,'s').astype(float)
which forces the time_interval to be seconds to align with the threshold inputs.
Also, it looks like the mapdates function within qartod.py will do some basic datetime sanitation but its only implemented on the Climatology tests.
When running the flat_line_test using a NcQcConfig object, passing in numpy datetime64[ns] time returns all "failed." This appears to be an issue between the numpy datetime64[ns] passed and the test expecting an array as datetime64[ms]. This should probably be checked when the tinp is passed to a test.