search

spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
147 stars 58 forks source link

Cluster load_csa() routine needs unit tests for each probe and datatype #717

Closed jameswilburlewis closed 3 months ago

jameswilburlewis commented 8 months ago

Cluster has all sorts of weird constructs in the CDFs we get from the CSA. In IDL, we've made a number of changes to spd_cdf2tplot to accommodate them, and there are probably similar issues lurking in pyspedas. (See https://github.com/spedas/pyspedas/issues/716, for example).

We need to add unit tests for all the datatypes we're getting from load_csa(), and verify that the variables can be loaded (or at least not cause crashes).

We may need to add postprocessing for some of them -- I know there are several cases where the FILLVALs don't match what's actually in the data arrays, so the spectrograms often come out all black when you try to plot them. This could possibly be addressed with master CDFs, but that's a bit unwieldy when we're extracting CDFs from a zip file that's created at their end.

jameswilburlewis commented 3 months ago

There are now tests for all the data types available via Cluster CSA. Some of them are being skipped at the moment, due to lack of data on the default dates, or server-side issues with the CIS data types.