ENH: Updates ICON code to remove warnings during load.

[rstoneback]

Description

Addresses #99

Addresses metadata warnings for ICON instruments. Requires https://github.com/pysat/pysat/pull/983

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Loaded IVM data jupyter notebook

Test Configuration

• Operating system: MacOS
• Version number: Python 3.9.7
• Any details about your local setup that are relevant

Checklist:

• [x] Make sure you are merging into the develop (not main) branch
• [x] My code follows the style guidelines of this project
• [x] I have performed a self-review of my own code
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] My changes generate no new warnings
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] New and existing unit tests pass locally with my changes
• [x] Any dependent changes have been merged and published in downstream modules
• [x] Add a note to CHANGELOG.md, summarizing the changes
• [x] Update zenodo.json file for new code contributors

[rstoneback]

Needs the in-progress pysat branch listed but it works for IVM. I think it should work for other ICON instruments as well.

[rstoneback]

pinging @aburrell given recent discussion about ICON and @jklenzing since he reported metadata issues with ICON to begin with.

[rstoneback]

This isn't quite right yet, got a little ahead. I'm wrapping of for weekend but I'm finding that array elements are being expanded out. Need to trace down where that is occurring and move it to after the meta_processor access.

From MIGHTI data, ignore the ivm label.

In [5]: ivm.meta.data.columns
Out[5]: 
Index(['FieldNam', 'LablAxis', 'Bin_Location', 'Valid_Range0', 'Valid_Range1',
       'Time_Base', 'Time_Scale', 'Var_Notes0', 'Var_Notes1', 'Var_Notes2',
       'Var_Notes3', 'Var_Notes4', 'Var_Notes5', 'Var_Notes6', 'Var_Notes7',
       'Var_Notes8', 'Var_Notes9', 'Var_Notes10', 'Var_Notes11', 'Var_Notes12',
       'Bin_Location0', 'Bin_Location1', 'Bin_Location2', 'units', 'long_name',
       'notes', 'desc', 'value_min', 'value_max', 'fill'],
      dtype='object')

Fill data is also pretty crazy here. Just saw this so not sure what it is about.

In [4]: ivm.meta.data
Out[4]: 
                                               FieldNam    LablAxis  ...                          value_max                                     fill
Epoch                                              Time        Time  ...                   31536000000000.0                   -9223372036854775808.0
Altitude                                       Altitude    Altitude  ...                             1000.0  9969209968386869046778552952102584320.0
Attitude_Conjugate_Maneuver                  Conj. Man.  Conj. Man.  ...                                1.0                                   -127.0
Attitude_LVLH_Normal                          LVLH Norm   LVLH Norm  ...                                1.0                                   -127.0
Attitude_LVLH_Reverse                          LVLH Rev    LVLH Rev  ...                                1.0                                   -127.0
Attitude_Limb_Pointing                         Att Limb    Att Limb  ...                                1.0                                   -127.0
Bin_Size                                       Bin Size    Bin Size  ...                              100.0                                   -127.0
Chi2                         Variance of each phase row       Chi^2  ...  1000000000000000019884624838656.0  9969209968386869046778552952102584320.0
Exposure_Time                             Exposure Time    Exp Time  ...                              120.0  9969209968386869046778552952102584320.0
Fringe_Amplitude                       Fringe Amplitude  Fringe Amp  ...  1000000000000000019884624838656.0  9969209968386869046778552952102584320.0
Fringe_Amplitude_Error           Fringe Amplitude Error     Amp Err  ...  1000000000000000019884624838656.0  9969209968386869046778552952102584320.0

[rstoneback]

Loading metadata is doing better here.

mighti = pysat.Instrument("icon", "mighti", tag="vector_wind_green", inst_id='vector')
mighti.load(2020, 1, use_header=True)
mighti.meta.data.columns

Index(['plot', 'axis', 'bin_loc', 'units', 'long_name', 'notes', 'desc',
       'value_min', 'value_max', 'fill'],
      dtype='object')

I have a bunch of tests and documentation to write over in pysat/pysat#983 but it is currently working on my system.

This is what happens when you copy from a jupyter notebook.

mighti.meta.data

plot | axis | bin_loc | units | long_name | notes | desc | value_min | value_max | fill
-- | -- | -- | -- | -- | -- | -- | -- | -- | --
Time | Time | 0.5 | ms | Sample time, average of A and B measurements. ... | A one-dimensional array defining the time dime... | Sample time, average of A and B measurements. ... | 0.0 | 31536000000000.0 | -1.0
Time | Time | 0.5 | ms | Sample time, midpoint of A and B measurements.... | See the notes for the variable Epoch. This var... | Sample time, midpoint of A and B measurements.... | 0.0 | 31536000000000.0 | -1.0
Altitude | Altitude | 0.5 | km | WGS84 altitude of each wind sample | A one-dimensional array defining the altitude ... | WGS84 altitude of each wind sample | 50.0 | 1000.0 | 9969209968386869046778552952102584320.0
Fringe Amplitude | Fringe Amp | 0.5 | arb | Fringe Amplitude | An approximate volume emission rate (VER) prof... | Fringe Amplitude

[jklenzing]

101 should fix the environment changes. We'll need to fix throughout the ecosystem.

[rstoneback]

GOLD instrument having issues with latest pysat. Waiting on merge of #110, then I'll revisit.