search

MillionConcepts / pdr

[P]lanetary [D]ata [R]eader - A single function to read all Planetary Data System (PDS) data into Python
Other
60 stars 6 forks source link

Load error values of some data when reading Juno/JADE files #50

Closed rbzheng closed 1 year ago

rbzheng commented 1 year ago

When I read this example in JNO-J_SW-JAD-5-CALIBRATED-V1.0 (please see ref1), I got the wrong value of 'DIM1_E' and right value of 'DATA' which have been tested. And there may be errors in other data, possibly due to displacement or bugs in CONTAINER type. Here is a test code.

import pdr
data=pdr.read('JAD_L50_HRS_ELC_TWO_DEF_2017086_V01.LBL')
tab=data['TABLE']

print(tab.loc[0,'DIM1_E_0':'DIM1_E_2047'].values.reshape(64,32)[:,0])
print(tab.loc[0,'DATA_0':'DATA_2047'].values.reshape(64,32)[:,0])

The 'DIM1_E' output is wrong and the 'DATA' output is right.

Output DIM1_E: [966518800000.0 894707100000.0 974645300000.0 930591870000.0
 837119700000.0 798567760000.0 897131200000.0 868507650000.0
 780258840000.0 771697740000.0 744121500000.0 669968960000.0
 498662050000.0 579350400000.0 520000670000.0 587698600000.0
 514104430000.0 365777940000.0 415876970000.0 439032280000.0
 318647760000.0 204029980000.0 269367790000.0 156951270000.0
 236470670000.0 233935060000.0 197731860000.0 93075470000.0 103446970000.0
 248979470000.0 140577670000.0 24597301000.0 99269420000.0 57017827000.0
 77155115000.0 56143553000.0 167358860000.0 140813880000.0 16594554000.0
 179233770000.0 22445095000.0 59724927000.0 186044100000.0 25854247000.0
 126756470000.0 43271512000.0 51427185000.0 -233773450000.0 143570810000.0
 53761340000.0 -37511377000.0 278001980000.0 9436561000.0 277946830000.0
 603936400000.0 370094800000.0 195210630000.0 145033380000.0
 166062000000.0 127271620000.0 104123605000.0 87407020000.0 -72188985000.0
 291688970000.0]

Output DATA: [1095216300000.0 1255912100000.0 1010507500000.0 981602600000.0
 1296541400000.0 1120366000000.0 1333463000000.0 1055353860000.0
 1089701400000.0 992039500000.0 1045064060000.0 985984470000.0
 957157700000.0 806583400000.0 664766450000.0 576056300000.0
 748781800000.0 558626100000.0 607873700000.0 667860000000.0
 487844100000.0 313727060000.0 370081000000.0 160760560000.0
 316935080000.0 221103670000.0 409881900000.0 387344760000.0
 218105300000.0 267263850000.0 95726390000.0 190925670000.0 230795870000.0
 27261618000.0 261343970000.0 109374960000.0 150559420000.0 115588825000.0
 272586960000.0 442547180000.0 244189760000.0 247328870000.0
...
 285688900000.0 856671800000.0 921093340000.0 1478896600000.0
 728130700000.0 550809200000.0 1193859900000.0 315433200000.0
 324642570000.0 808213500000.0 841204600000.0 -19234296000.0
 511574900000.0]

But when I only keep DATA and DIM1_E in the *.LBL file, then change the CONTAINER to COLUMN, and match the ITEMS, START_BYTE and BYTES:

OBJECT              = COLUMN                                              
      NAME              = DATA                                                
      DATA_TYPE         = PC_REAL                                             
      START_BYTE        = 315                                                   
      ITEMS             = 2048                                                   
      ITEM_BYTES        = 4                                                   
      BYTES             = 8196                                                   
      /* if background removed, VALID_MINIMUM, can be <0 */                   
      VALID_MINIMUM     = -5.00e+14 /* = -999998/2e-09 */                     
      VALID_MAXIMUM     =  1.12e+15 /* = 2250000/2e-09 */                     
      MISSING_CONSTANT  = -999999                                             
      UNIT              = "1/(m^2 sr s)"                                      
      DESCRIPTION       = "DATA: Differential Energy Flux (SI units)          
                           64 Energy x 32 Look Directions.                    
                           "                                                  
/* RJW, DATA, f, 2, 64, 32 */                                                 
END_OBJECT          = COLUMN                                              

OBJECT              = COLUMN                                              
      NAME              = DIM1_E                                              
      DATA_TYPE         = PC_REAL                                             
      START_BYTE        = 33083                                                   
      ITEMS             = 2048                                                   
      ITEM_BYTES        = 4                                                   
      BYTES             = 8196                                                   
      VALID_MINIMUM     =     0.0                                             
      VALID_MAXIMUM     = 99000.0 /* Rounded up to whole keV/q */             
      MISSING_CONSTANT  = 99999.0                                             
      UNIT              = "eV/q"                                              
      DESCRIPTION       = "1st Dimension of DATA: Energy (center) in eV/q."   
/* RJW, DIM1_E, f, 2, 64, 32 */                                               
END_OBJECT          = COLUMN

the two outputs are right.

Output DIM1_E: [  100.211   112.076   124.028   139.064   154.895   172.321   191.798
   214.574   238.562   266.321   296.413   331.753   368.269   411.227
   458.062   510.549   569.014   635.272   707.082   789.524   879.737
   981.646  1092.53   1219.075  1356.921  1515.487  1687.378  1884.479
  2095.648  2342.841  2604.627  2950.948  3253.325  3629.974  4047.429
  4508.255  5016.57   5604.032  6227.522  6967.207  7745.898  8636.086
  9621.097 10739.074 11953.525 13315.562 14831.049 16562.906 18428.473
 20575.008 22919.393 25552.047 28454.154 31750.922 35389.742 39698.477
 43966.082 49003.953 54622.633 60917.27  67910.25  75671.06  84416.76
 94060.42 ]

Output DATA: [ 1.09521627e+12  1.25591211e+12  1.01050751e+12  9.81602599e+11
  1.29654142e+12  1.12036597e+12  1.33346296e+12  1.05535386e+12
  1.08970141e+12  9.92039469e+11  1.04506406e+12  9.85984467e+11
  9.57157671e+11  8.06583402e+11  6.64766448e+11  5.76056328e+11
  7.48781830e+11  5.58626111e+11  6.07873729e+11  6.67860009e+11
  4.87844086e+11  3.13727058e+11  3.70081006e+11  1.60760562e+11
  3.16935078e+11  2.21103669e+11  4.09881903e+11  3.87344761e+11
  2.18105299e+11  2.67263853e+11  9.57263872e+10  1.90925668e+11
  2.30795870e+11  2.72616182e+10  2.61343969e+11  1.09374964e+11
  1.50559425e+11  1.15588825e+11  2.72586965e+11  4.42547175e+11
  2.44189757e+11  2.47328866e+11  3.73126103e+11  6.36105564e+10
  3.66657929e+11  2.93417419e+11  5.06448970e+11 -8.73576694e+10
  3.62037182e+11  6.81055617e+11  4.51320676e+11  2.85688889e+11
  8.56671781e+11  9.21093341e+11  1.47889665e+12  7.28130716e+11
  5.50809174e+11  1.19385987e+12  3.15433189e+11  3.24642570e+11
  8.08213479e+11  8.41204630e+11 -1.92342958e+10  5.11574901e+11]

In addition, I test the L30 file at the same time record (see ref 2). There are also wrong 'DIM1_E' value and right 'DATA' value.

Here are references:

rbzheng commented 1 year ago

The pdr version I used is the latest "1.0.1".

AlimyBreak commented 1 year ago

wow，逛这里才发现级别5的数据已经公布了。

m-stclair commented 1 year ago

problem was caused by an error in offset tracking when a CONTAINER's first child is another CONTAINER. closed by 2522f06. ty for the very useful bug report.