search

spedas / bleeding_edge

IDL-based Space Physics Environment Data Analysis Software (bleeding edge)
http://www.spedas.org
Other
7 stars 0 forks source link

MMS FEEPS test failures #78

Open ericthewizard opened 1 year ago

ericthewizard commented 1 year ago

Two of the FEEPS tests are failing:

"All tests" test suite starting (1 test suite/case, 59 tests) "mms_load_feeps_ut" test case starting (59 tests) test_active_eyes_after_update: passed (0.001456 seconds) test_active_eyes_before_update: passed (0.000705 seconds) test_active_eyes_brst: passed (0.000074 seconds) test_active_eyes_sitl_after_updates: passed (0.003113 seconds) test_active_eyes_sitl_before_updates: passed (0.000427 seconds) test_bad_eyes_brst_probe1: passed (floating-point operand error) (13.220033 seconds) test_bad_eyes_brst_probe2: passed (floating-point operand error) (14.888487 seconds) test_bad_eyes_brst_probe3: passed (floating-point operand error) (15.340245 seconds) test_bad_eyes_brst_probe4: passed (floating-point operand error) (16.474968 seconds) test_bad_eyes_srvy_probe2: passed (floating-point operand error) (12.983074 seconds) test_bad_eyes_srvy_probe3: passed (floating-point operand error) (13.125501 seconds) test_bad_eyes_srvy_probe4: passed (floating-point operand error) (13.685164 seconds) test_bad_lower_channels_brst: passed (floating-point operand error) (295.808878 seconds) test_bad_lower_channels_brst_suffix: passed (floating-point operand error) (287.575444 seconds) test_bad_lower_channels_srvy_sitl: failed "Subscripts are not allowed with variable attributes." (floating-point operand error) (1.244252 seconds) test_brst_caps_pad: passed (floating-point operand error) (13.178688 seconds) test_brst_pad: passed (floating-point operand error) (13.474562 seconds) test_energy_channel_brst_probe1: passed (floating-point operand error) (12.626221 seconds) test_energy_channel_brst_probe1_suffix: passed (floating-point operand error) (12.576775 seconds) test_energy_channel_brst_probe2: passed (floating-point operand error) (14.579808 seconds) test_energy_channel_brst_probe3: passed (floating-point operand error) (15.067798 seconds) test_energy_channel_brst_probe4: passed (floating-point operand error) (15.826989 seconds) test_energy_channel_srvy_probe1: passed (floating-point operand error) (16.520419 seconds) test_energy_channel_srvy_probe2: passed (floating-point operand error) (12.288089 seconds) test_energy_channel_srvy_probe3: passed (floating-point operand error) (12.692862 seconds) test_energy_channel_srvy_probe4: passed (floating-point operand error) (13.616869 seconds) test_ion_bad_eyes_brst: passed (floating-point operand error) (4.137963 seconds) test_ion_bad_eyes_srvy: passed (floating-point operand error) (4.929118 seconds) test_last_pad_bin: passed (floating-point operand error) (18.108450 seconds) test_load_ion: passed (floating-point operand error) (5.697390 seconds) test_load_ion_brst: passed (3.852089 seconds) test_load_l1a_data: passed (3.784339 seconds) test_load_l1a_multi_datatypes: passed (0.676009 seconds) test_load_l1a_set_bad_datatype: passed (3.883852 seconds) test_load_l1a_set_good_datatype: passed (0.397180 seconds) test_load_l1b: passed (floating-point operand error) (7.103697 seconds) test_load_l1b_ion: passed (floating-point operand error) (2.024462 seconds) test_load_l2: passed (0.000694 seconds) test_load_l2_multi_datatypes: passed (floating-point operand error) (31.726617 seconds) test_load_multi_probe: passed (floating-point operand error) (164.927689 seconds) test_load_sitl_omni: failed "Problem loading omni-directional FEEPS spectra using SITL files" (floating-point operand error) (0.688281 seconds) test_load_spdf: passed (floating-point operand error) (11.734249 seconds) test_load_suffix_pad: passed (floating-point operand error) (8.963675 seconds) test_load_timeclip: passed (11.335058 seconds) test_omni_not_replacing_0s_with_nans: passed (floating-point operand error) (17.309480 seconds) test_omni_offbyone_err: passed (floating-point operand error) (16.796101 seconds) test_pad: passed (floating-point operand error) (18.210312 seconds) test_pad_angles_from_bfield: passed (floating-point operand error) (20.801511 seconds) test_pad_bad_en: passed (floating-point operand error) (3.232583 seconds) test_pad_binsize: passed (floating-point operand error) (25.015308 seconds) test_pad_limited_en: passed (floating-point operand error) (4.032120 seconds) test_pad_low_en: passed (floating-point operand error) (16.551854 seconds) test_smooth: passed (floating-point operand error) (16.682553 seconds) test_smooth_pad: passed (floating-point operand error) (21.699084 seconds) test_suffix: passed (floating-point operand error) (11.180172 seconds) test_time_dependent_sun_masks: passed (0.150068 seconds) test_tt2000: passed (floating-point operand error) (4.784470 seconds) test_varformat_brst_count_rate: passed (4.749781 seconds) test_varformat_brst_intensity: passed (floating-point operand error) (4.375304 seconds) Test coverage: 88.2% Untested lines mms_load_feeps: lines 181-187, 189, 191-197, 199, 201, 203-204 mms_feeps_omni: lines 114-115, 117-121 mms_feeps_pad_spinavg: lines 51-52, 63-64 mms_feeps_pad: lines 94-95 mms_feeps_pitch_angles: lines 33-34, 48-49, 51, 53-57, 252-253, 257, 260, 263, 267, 270, 273, 277-279, 281-282, 284-285, 287-288, 290-291, 293-294, 297, 300-311, 313-315, 317 mms_feeps_remove_bad_data: lines 251-254, 260-262, 267-270 mms_feeps_energy_table: lines 30-31, 34-35, 39-40 Completely covered routines mms_feeps_remove_sun, mms_feeps_smooth, mms_feeps_spin_avg, mms_feeps_split_integral_ch, mms_feeps_correct_energies, mms_feeps_flat_field_corrections, mms_read_feeps_sector_masks_csv, mms_feeps_active_eyes Results: 57 / 59 tests passed, 0 skipped Results: 57 / 59 tests passed, 0 skipped (88.2% coverage)

jameswilburlewis commented 1 year ago

Same two tests still failing:

(base) Jamess-MBP:~ jwl$ grep failed mms_feeps_results
    test_bad_lower_channels_srvy_sitl: failed "Subscripts are not allowed with variable attributes." (floating-point operand error) (3.152932 seconds)
    test_load_sitl_omni: failed "Problem loading omni-directional FEEPS spectra using SITL files" (floating-point operand error) (16.495555 seconds)
jameswilburlewis commented 11 months ago

The problem can be reproduced with the command:

mms_load_feeps, probes=[1], level='sitl', trange=['2016-11-01', '2016-11-02']

Here are some log messages:

MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_count_rate_sensorid_5 MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_count_rate_sensorid_11 MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_count_rate_sensorid_12 % Compiled module: MMS_FEEPS_REMOVE_SUN. MMS_FEEPS_REMOVE_SUN(33): Error - couldn't find the spin sector variable!!!! Cannot remove sun contamination! % Compiled module: MMS_FEEPS_OMNI. % Compiled module: MMS_FEEPS_SPIN_AVG. MMS_FEEPS_SPIN_AVG(33): Error, couldn't find the tplot variable containing the spin sectors for calculating the spin averages. MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_intensity_sensorid_5 MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_intensity_sensorid_11 MMS_FEEPS_SPLIT_INTEGRAL_CH(38): Couldnt find the variable: mms1_epd_feeps_srvy_sitl_electron_top_intensity_sensorid_12 MMS_FEEPS_REMOVE_SUN(33): Error - couldn't find the spin sector variable!!!! Cannot remove sun contamination! MMS_FEEPS_SPIN_AVG(33): Error, couldn't find the tplot variable containing the spin sectors for calculating the spin averages.

What we've found so far is that the data available at the SDC is a mix of different CDF file versions on different dates, for example: mms1_feeps_srvy_sitl_electron_20161101000000_v7.1.1.cdf mms1_feeps_srvy_sitl_electron_20171101000000_v6.0.10.cdf mms1_feeps_srvy_sitl_electron_20181101000000_v6.0.7.cdf mms1_feeps_srvy_sitl_electron_20221101000000_v7.2.4.cdf

If we pick a date with version 6 CDF data available, mms_load_feeps seems to work fine. It fails for any date with version 7 CDFs. There are variables in the version 6 CDFs that aren't present in version 7, and that's why the test fails for those dates.