fhs / pyhdf

Python wrapper around the NCSA HDF version 4 library
https://pypi.org/project/pyhdf/
Other
46 stars 15 forks source link

ACE Level 2 Data HDFv4.1r2 #33

Closed htyeim closed 5 years ago

htyeim commented 5 years ago

Hello, I am new to HDF data format. I am now dealing with some data which is saved in HDFv4.1r2. Here is the data description: http://www.srl.caltech.edu/ACE/ASC/hdf.html.

Here is a link to a real data file: https://cdaweb.gsfc.nasa.gov/pub/data/ace/magswe/level2_hdf/64sec/magswe_data_64sec_2246.hdf

When I use the code below to read the file:

from pyhdf.SD import SD
filename = 'magswe_data_64sec_2246.hdf'

FILE_NAME = filename
hdf = SD(FILE_NAME)

print(hdf.datasets())

I only got an empty data set.

{}

I wound if pyhdf could read this type of data correctly.

And I think there is a repository which might contain some related information: https://github.com/jimsrc/pyhdf_ace.

Thank you for your time!

fhs commented 5 years ago

It looks like that ACE HDF file is using Vdata, not SD. The hdp command (which comes with HDF4) is handy when exploring an unknown HDF file. For example:

$ /opt/hdf4/bin/hdp dumpvd -h magswe_data_64sec_2246.hdf
...
Vdata: 0
   tag = 1962; reference = 3;
   number of records = 36450; interlace = FULL_INTERLACE (0);
   fields = [year, day, hr, min, sec, fp_year, fp_doy, ACEepoch, Np, Tp, 
                  Alpha_ratio, Vp, V_rtn_r, V_rtn_t, V_rtn_n, V_gse_x, V_gse_y, 
                  V_gse_z, V_gsm_x, V_gsm_y, V_gsm_z, B_rtn_r, B_rtn_t, B_rtn_n, 
                  B_gse_x, B_gse_y, B_gse_z, B_gsm_x, B_gsm_y, B_gsm_z, Bmag, 
                  Lambda, Delta, dBrms, pos_gse_x, pos_gse_y, pos_gse_z, 
                  pos_gsm_x, pos_gsm_y, pos_gsm_z, pos_hs_x, pos_hs_y, 
                  pos_hs_z, MAG_pts];
   record size (in bytes) = 188;
   name = MAGSWE_data_64sec; class = <Undefined>;
Vdata: 1
   tag = 1962; reference = 1781;
   number of records = 27; interlace = FULL_INTERLACE (0);
   fields = [record_number, year, day, days_jan_1_1996];
   record size (in bytes) = 16;
   name = level2_record_doy; class = level2_record DOY table;

Take a look at the file format description and an IDL implementation. You should be able to implement a reader based on pyhdf, just like the IDL implementation.