pysat / pysatNASA

pysat support for NASA Instruments
BSD 3-Clause "New" or "Revised" License
21 stars 6 forks source link

BUG: `test_load_multiple_days` failures #218

Open jklenzing opened 11 months ago

jklenzing commented 11 months ago

Description

pysat 3.2.0 has a new universal test: test_load_multiple_days. Several instruments fail this test

To Reproduce this bug:

run pytest with pysat develop

Test configuration

Additional context

jklenzing commented 11 months ago

Some of these are cause by an issue with CDAS download routines, where the localize function adds a timestamp of '00:00:00' to a date, meaning only the first day will be returned rather than two days. BUGFIX downstream of #198

https://github.com/pysat/pysatNASA/blob/4f0df4667aa8112f9cee39a4d61f8cd0c72b42b5/pysatNASA/instruments/methods/cdaweb.py#L965-L967

Using pysat.utils.time.filter_datetime_inputinstead fixes this problem for some of the data products. Fixes 15 of the above errors

jklenzing commented 11 months ago

There's also an issue if data files do not begin exactly at zero seconds. CDAS WS interprets the end date as a beginning of day, so if data starts at 00:00:01 it will ignore the file. Adding 86399 seconds to the end date fixes this for another 5 instruments.

jklenzing commented 11 months ago

DE2 VEFI fixed by #217. New tests run when those changes are pulled in

jklenzing commented 9 months ago

TODO:

jklenzing commented 7 months ago

The remaining issues (icon_euv, maven_insitu_kp) are basically the same => the datasets have covariance matrices with duplicate dimension names, ie dims = (time, compno_3, compno_3). Will write up a new issue.