search

pysat / pysatCDF

Python reader for NASA CDF file format
BSD 3-Clause "New" or "Revised" License
10 stars 7 forks source link

Expand read support for higher dimension data #16

Open rstoneback opened 4 years ago

rstoneback commented 4 years ago

pysatCDF currently uses a Fortran interface to the underlying CDF library. Each time a variable is loaded the code needs to open the file, read the variable, then close the file. Each variable type requires its own function. To reduce read time, all variables of a given type and dimensionality (<=2) are all read at once. The memory passed to Fortran is sized based off of the largest dimensions, so loading 1D and 10D data together is inefficient.

Convert the raw fortran read routines to support the highest CDF dimension (10?). Keep current loads of 1D and 2D together. Then load 3D, 4D, 5D, as needed, individually.

rstoneback commented 4 years ago

This will enable pysat to load all NASA data.

asherp commented 4 years ago

Is there an example file I can test on?