search

irfu / Lapdog_GIT

Matlab program for generating Rosetta RPCLAP calibrated-level & derived-level data from pds-generated data
BSD 2-Clause "Simplified" License
0 stars 1 forks source link

NED.TAB quality values >1.0 #32

Closed ErikPGJ closed 5 years ago

ErikPGJ commented 5 years ago

Found in PRL-EXT3 (all comet phases). Not found in ASTx.

Checks (assertions) in createLBL code indicates that many NED.TAB files are badly formatted, in practice the quality value columns is too wide due to too high values (qv>=10). (Have not found any NaN.)

Example: Error using EJ_library.ro.create_OBJTABLE_LBL_file>warning_error___LOCAL (line 736) TAB file appears to not have rows of uniform length. File size is NOT an integer multiple of the first/last rows length: File: "/data/rosetta/PDS_Archives/DATASETS/delivery/delivery_2019-06-30_DERIV_science_mp/gen_2019-04-17/gen_2019-06-20/RO-C-RPCLAP-5-ESC1-DERIV-V2.0/2014/NOV/D19/RPCLAP_20141119_000000_515_NED.TAB" fileSize = 39434 nBytesPerRow = 73

Error in EJ_library.ro.create_OBJTABLE_LBL_file (line 237) warning_error___LOCAL(Exc, Settings.tabLblInconsistencyPolicy)

Error in createLBL.create_LBL_files (line 374) EJ_library.ro.create_OBJTABLE_LBL_file(...

Error in createLBL (line 180) createLBL.create_LBL_files(Clfd)

Error in lapdog (line 122) createLBL(1,1)

Error in erikpgjohansson.ro.run_lapdog>@()lapdog(calib1Path,missionPhaseNameAbbrev,pdsMissionCalendar) (line 60) funcPtr = @() lapdog(calib1Path, missionPhaseNameAbbrev, pdsMissionCalendar);

Error in erikpgjohansson.ro.lapdog_convention_wrapper>call_function_pointer (line 207) funcPtr();

Error in erikpgjohansson.ro.lapdog_convention_wrapper (line 188) call_function_pointer(funcPtr)

Error in erikpgjohansson.ro.run_lapdog (line 62) erikpgjohansson.ro.lapdog_convention_wrapper(funcPtr, metakernel, lapdogDir, ...

Excerpt from file /data/rosetta/PDS_Archives/DATASETS/delivery/delivery_2019-06-30_DERIV_science_mp/gen_2019-04-17/gen_2019-06-20/RO-C-RPCLAP-5-ESC1-DERIV-V2.0/2014/NOV/D19/RPCLAP_20141119_000000_515_NED.TAB in error message above:

2014-11-19T17:37:39.292, 375039387.935578, 3.3744981e+01, 0.80, 3, 000 2014-11-19T17:40:19.292, 375039547.935578, 2.9860399e+01, 0.80, 3, 000 2014-11-19T17:42:59.292, 375039707.935578, 2.4494342e+01, 0.89, 3, 000 2014-11-19T17:45:39.292, 375039867.935578, 1.4244811e+01, 26.28, 3, 000 2014-11-19T17:48:19.306, 375040027.949231, 1.9454799e+01, 1.52, 3, 000 2014-11-19T17:50:59.306, 375040187.949231, 3.4352022e+01, 0.80, 3, 000 2014-11-19T17:53:39.306, 375040347.949231, 1.8653615e+01, 1.73, 3, 000 2014-11-19T17:56:19.306, 375040507.949231, 3.5685189e+01, 0.80, 3, 000 2014-11-19T17:58:59.306, 375040667.949231, 1.7132083e+01, 2.45, 3, 000 2014-11-19T18:01:39.306, 375040827.949231, 3.1777207e+01, 0.80, 3, 000 2014-11-19T18:04:19.306, 375040987.949231, 1.9180620e+01, 1.58, 3, 000 2014-11-19T18:06:59.293, 375041147.935578, 1.9707943e+01, 1.46, 3, 000 2014-11-19T18:09:39.293, 375041307.935578, 1.9378049e+01, 1.53, 3, 000 2014-11-19T18:12:19.293, 375041467.935578, 2.2583429e+01, 1.05, 3, 000 2014-11-19T18:14:59.293, 375041627.935578, 2.4617584e+01, 0.89, 3, 000 2014-11-19T18:17:39.293, 375041787.935578, 2.4592785e+01, 0.88, 3, 000 2014-11-19T18:20:19.293, 375041947.935578, 1.9581630e+01, 1.47, 3, 000 2014-11-19T18:22:59.293, 375042107.935578, 1.4740341e+01, 8.81, 3, 000 2014-11-19T18:25:39.307, 375042267.949231, 1.3440763e+01, 13.07, 3, 000 2014-11-19T18:28:19.307, 375042427.949231, 1.2906966e+01, 6.14, 3, 000 2014-11-19T18:30:59.307, 375042587.949231, 1.9328503e+01, 1.54, 3, 000 2014-11-19T18:33:39.307, 375042747.949231, 1.2679802e+01, 4.96, 3, 000 2014-11-19T18:36:19.307, 375042907.949231, 1.3022819e+01, 7.00, 3, 000 2014-11-19T18:38:59.307, 375043067.949231, 1.4331176e+01, 16.85, 3, 000 2014-11-19T18:41:39.307, 375043227.949231, 1.4279987e+01, 18.90, 3, 000 2014-11-19T18:44:19.294, 375043387.935578, 1.4294315e+01, 18.15, 3, 000 2014-11-19T18:46:59.294, 375043547.935578, 1.3059345e+01, 7.45, 3, 000 2014-11-19T18:49:39.294, 375043707.935578, 1.2980978e+01, 6.81, 3, 000 2014-11-19T18:52:19.294, 375043867.935578, 1.1634614e+01, 2.37, 3, 000 2014-11-19T18:54:59.294, 375044027.935578, 1.1572246e+01, 2.30, 3, 000 2014-11-19T18:57:39.294, 375044187.935578, 1.2693658e+01, 5.09, 3, 000 2014-11-19T19:00:19.294, 375044347.935578, 1.4831221e+01, 7.49, 3, 000 2014-11-19T19:02:59.294, 375044507.935578, 1.2528243e+01, 4.45, 3, 000 2014-11-19T19:05:39.308, 375044667.949231, 1.2033507e+01, 2.98, 3, 000 2014-11-19T19:08:19.308, 375044827.949231, 1.2667219e+01, 5.05, 3, 000 2014-11-19T19:10:59.308, 375044987.949231, 1.4227863e+01, 19.83, 3, 000 2014-11-19T19:13:39.308, 375045147.949231, 1.4237213e+01, 19.18, 3, 000 2014-11-19T19:16:19.308, 375045307.949231, 1.1683277e+01, 2.47, 3, 000 2014-11-19T19:18:59.308, 375045467.949231, 1.4594236e+01, 9.66, 3, 000 2014-11-19T19:21:39.308, 375045627.949231, 1.4842320e+01, 7.24, 3, 000 2014-11-19T19:24:19.308, 375045787.949231, 1.4415957e+01, 12.59, 3, 000 2014-11-19T19:26:59.294, 375045947.935578, 3.2838200e+01, 0.80, 3, 000 2014-11-19T19:29:39.294, 375046107.935578, 3.4368074e+01, 0.80, 3, 000

frejon commented 5 years ago

Untested bugfix here https://github.com/irfu/Lapdog_GIT/commit/0eeba495bfcd213dad3531d492fd31de7b97b9c7 I accidentally wrote the wrong issue # in the commit title...

ErikPGJ commented 5 years ago

Worked for rerun (analysis) of EXT3 (+AST1, AST2). Further rerun tests in progress.

Could use assertion before writing quality values to disks if problem persists, e.g. assert(any((qv>1.0) || (qv<0.0)))

ErikPGJ commented 5 years ago

And the assertion should be inverted of course... assert(all((qv<=1.0) && (qv>=0.0)))

ErikPGJ commented 5 years ago

Seems fixed, although technically not tested on all science phases at time of writing.