search

spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
143 stars 58 forks source link

mms_part_getspec tests fail with new version of cdflib #812

Closed jameswilburlewis closed 3 months ago

jameswilburlewis commented 3 months ago

The new cdflib seems to be returning different values than before (probably nans?) in some situation when reading a CDF. Many/most of the pyspedas test suites tests pass, but the tests for mms_part_getspec fail with errors like this:

Traceback (most recent call last):
  File "/home/runner/work/pyspedas/pyspedas/pyspedas/mms/tests/mms_part_getspec.py", line 50, in test_fpi_brst_e
    mms_part_getspec(trange=['2015-10-16/13:06:07', '2015-10-16/13:06:08'],
  File "/home/runner/work/pyspedas/pyspedas/pyspedas/mms/particles/mms_part_getspec.py", line 212, in mms_part_getspec
    new_vars = mms_part_products(tname, species=species, instrument=instrument, probe=prb, data_rate=data_rate,
  File "/home/runner/work/pyspedas/pyspedas/pyspedas/mms/particles/mms_part_products.py", line 356, in mms_part_products
    fac_data = spd_pgs_regrid(fac_data, regrid)
  File "/home/runner/work/pyspedas/pyspedas/pyspedas/particles/spd_part_products/spd_pgs_regrid.py", line 58, in spd_pgs_regrid
    data_grid_interp = griddata(phi_temp, theta_temp, data_temp)
  File "/home/runner/work/pyspedas/pyspedas/pyspedas/particles/spd_part_products/spd_pgs_regrid.py", line 77, in griddata
    return NearestNDInterpolator(points, data)
  File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/scipy/interpolate/_ndgriddata.py", line 98, in __init__
    self.tree = cKDTree(self.points, **tree_options)
  File "_ckdtree.pyx", line 564, in scipy.spatial._ckdtree.cKDTree.__init__
ValueError: data must be finite, check for nan or inf values
jameswilburlewis commented 3 months ago

I'll need to track down what variable or attribute is getting read differently, then I'll be able to open an issue with the cdflib maintainers.

jameswilburlewis commented 3 months ago

It looks like the old version of cdflib was converting time variables to datetime.datetime objects, while the new version returns np.datetime64. So there has to be different code to implement centered_measurement delta_t values (converting to datetime.timedelta or np.timedelta64 respectively).

Retesting with pytplot-mpl-temp 2.2.13, fingers crossed....