pysat / pysatNASA

pysat support for NASA Instruments
BSD 3-Clause "New" or "Revised" License
21 stars 6 forks source link

ENH: ACE data #140

Closed jklenzing closed 1 year ago

jklenzing commented 1 year ago

Description

Addresses #127

Adds data for ACE L2 data. L1 data is managed at pysatSpaceWeather, so these need unique names to avoid messing with the registry. Adding '_l2' to each name to signify Level 2.

Type of change

How Has This Been Tested?

Tested locally via pytest.

Test Configuration

Checklist:

jklenzing commented 1 year ago

Pinging @aburrell for a review

jklenzing commented 1 year ago

Why is test.nc included?

test.nc was accidentally committed. Removed.

aburrell commented 1 year ago

Ok, I just went to play with the data, and there's no information on which of the inst_id values I should pick. This could be in the module docstrings or (if more appropriate) these could be tags instead of inst_ids and have little descriptions.

jklenzing commented 1 year ago

I like the idea of changing these to tags. I have yet to find descriptions of the products in the online docs. I'll poke around a little more.

aburrell commented 1 year ago

Comparing the CDF and HDF directory structures I'm going to guess it's related to the 1day, 1hr, 3min, 12sec, and 27day products?

jklenzing commented 1 year ago

Comparing the CDF and HDF directory structures I'm going to guess it's related to the 1day, 1hr, 3min, 12sec, and 27day products?

Looking at the descriptions on the main page(https://cdaweb.gsfc.nasa.gov/, search for ACE), that matches the descriptions. I'll update the tag names with the cadence since h1, k0, etc seems to be an internal SPDF organization thing rather than part of the data product.

jklenzing commented 1 year ago

Here's my current proposed structure:

platform = 'ace'
name = 'epam_l2'
tags = {'level2': 'ACE/EPAM Solar Energetic Particle Level 2 Data',
        'key': 'ACE/EPAM Solar Energetic Particle Key Parameters'}
inst_ids = {'12sec': ['level2'],
            '5min': ['key', 'level2'],
            '1hr': ['key', 'level2']}

The 'h' from the old version refers to the level 2 data, 'k' to 'key parameters'. Note that the key parameter product does not exist for every time cadence.

I have a working version for EPAM, but ultimately this will be easier if #145 is finished and merged in, since we can pull remote file names in directly from cdasws. Pinging @JonathonMSmith for thoughts.

jklenzing commented 1 year ago

Fixing the ref / property order here so I can clean up the rest of the code in parallel.

aburrell commented 1 year ago

The metadata is a bit messed up. There are double values for 'fill', for instance, which could be cleaned up to make the data easier to handle:

swepam.meta['Np']

units                                                         #/cc
long_name                                                H_Density
notes                                                             
desc                                                              
value_min                                                      NaN
value_max                                                      NaN
fill                                                           NaN
plot_label                                                        
axis                                                              
scale                                                       linear
FIELDNAM                                        Proton No. density
DICT_KEY                                            density>number
CATDESC                   Solar Wind Proton Number Density, scalar
standard_name                                   Proton No. density
VALIDMIN0                                                     -1.0
VALIDMIN1                                                     -1.0
VALIDMIN2                                                     -1.0
VALIDMAX0                                                     -1.0
VALIDMAX1                                                     -1.0
VALIDMAX2                                                     -1.0
SCALEMIN0                                                     -1.0
SCALEMIN1                                                     -1.0
SCALEMIN2                                                     -1.0
SCALEMAX0                                                     -1.0
SCALEMAX1                                                     -1.0
SCALEMAX2                                                     -1.0
FORM_PTR                                                          
LABL_PTR_1                                                        
UNIT_PTR                                                          
FILLVAL                                              -2147483648.0
SCALETYP                                                          
VALIDMIN                                                       0.0
VALIDMAX                                                     200.0
SCALEMIN                                                       0.0
SCALEMAX                                                     100.0
LABLAXIS                                                 H_Density
AVG_TYPE                                                          
VAR_NOTES        Np is the proton number density in units of cm...
children                                                      None
Name: Np, dtype: object
jklenzing commented 1 year ago

Updated with the meta_translation to convert the metadata.

jklenzing commented 1 year ago

Created a standard clean routine to pass through to each ace instrument. Ready for re-review.