spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
149 stars 58 forks source link

ERG load routines doing their own CDF processing? #781

Closed jameswilburlewis closed 2 months ago

jameswilburlewis commented 7 months ago

It looks like erg.sd_fit() and erg.isee_brio() are loading some data the usual way, with cdf_to_tplot, then opening one of the CDFs and doing some sort of position table post-processing. We should see if this can be replaced with some sort of manipulation of already-loaded tplot variables, or if this requires another pass over the CDF to do something cdf_to_tplot doesn't or can't support.

camera_omti_asi seems to read the CDF to change some data types back to various types of integers after calling clip(), which can turn the data arrays into floats apparently (because the clipped data gets replaced with NaN?). Maybe we need an option to clip to leave integer data alone, or specify an alternate fill value, or modify the omti_asi code to only clip where necessary or skip the non-floating-point variables.

isee_brio() seems to be trying to do FILLVAL processing on its own -- maybe this didn't work right in an older version of cdf_to_tplot?

gmag_isee_induction() is reading a CDF to build some kind of frequency-dependent structure

isee_vlf() is reading a CDF to get parameters for a gain calibration

erg_hep_get_dist() is reading a CDF to get an energy array -- could this already exist in the metadata?

erg_lepe_get_dist() reads a CDF to get an angle array -- already in metadata somewhere?

erg.load() reads a CDF and seems to be doing something with prefixes and suffixes for the 'notplot' case

jameswilburlewis commented 2 months ago

Many of these routines work the same way in IDL SPEDAS. I think we'll leave them as-is unless there's a compelling reason to change.