spedas / bleeding_edge

IDL-based Space Physics Environment Data Analysis Software (bleeding edge)
http://www.spedas.org
Other
7 stars 0 forks source link

tplot2cdf missing some variable attributes #37

Closed jameswilburlewis closed 1 year ago

jameswilburlewis commented 1 year ago

The UNITS and COORDINATE_SYSTEM variable attributes should be set in the CDF if the corresponding entries are present in the data_att dlimits structure for the tplot variable. This would be very useful for unit testing, especially when validating Python results against IDL results.

xandrd commented 1 year ago

UNITS attribute is supported by the tplot2cdf. You can store the pointer to the attribute structure via s.CDF.VARS.attrptr.

Could you give an example (a crib) when COORDINATE_SYSTEM are included into the tplot variable?

xandrd commented 1 year ago

Note, you can add COORDINATE_SYSTEM with the current functionality:

; Adding cdf structure
tplot_add_cdf_structure, 'array_variable'
get_data, 'array_variable', limits=s

cdf_y_attr_struct = *s.CDF.VARS.attrptr

; Adding units
cdf_y_attr_struct.UNITS = 'arb. unit.'

; Adding coordinate system (`str_element` must be used because it is not a default field)
str_element,cdf_y_attr_struct,'COORDINATE_SYSTEM','arb. coord. system.',/add

; Save CDF structure into tplot variable
s.CDF.VARS.attrptr = ptr_new(cdf_y_attr_struct)
options,'array_variable','CDF', s.CDF

; Save cdf file
tplot2cdf, filename='example_array_variable_w_attibutes', tvars='array_variable'
xandrd commented 1 year ago

@jameswilburlewis, could you provide a crib sheet or an example of saving coordinate system and units into the tplot variable?

Is there a general rule how we should deal with metadata in tplot variables?

jameswilburlewis commented 1 year ago

Here's a test routine illustrating the issue:

pro repro_data_att
   timespan,'2007-03-23',1,/day
   thm_load_state,probe='a'
   get_data,'tha_state_pos',dl=dl
   help,dl.data_att  ; initialized when data is loaded 
   tplot2cdf,filename='tp_data_att_test.cdf',tvars=['tha_state_pos'],/default_cdf_structure
   del_data,'*'
   spd_cdf2tplot,'tp_data_att_test.cdf'
   get_data,'tha_state_pos',dl=dl
   help,dl.cdf.vatt.units
   help,dl.cdf.vatt.coordinate_system  ; missing, should be copied from data_att.coord_sys of stored variable
end

Last few lines of output:

TPLOT2CDF_SAVE_VARS(64): Open CDF file tp_data_att_test.cdf (id :   624444720)
TPLOT2CDF_SAVE_VARS(216): CDF file tp_data_att_test.cdf is closed
STORE_DATA(180): Deleted            2 variables
SPD_MMS_CDF_LOAD_VARS(232): Loading file: "tp_data_att_test.cdf"
STORE_DATA(314): Creating tplot variable: 1 tha_state_pos
<Expression>    STRING    = 'km'
% Tag name COORDINATE_SYSTEM is undefined for structure <Anonymous>.
% Execution halted at: REPRO_DATA_ATT     11 C:\Users\james\Desktop\spdsoft_trunk\projects\themis\spin\repro_data_att.pro
%                      $MAIN$          

If data_att.coord_sys is present in the saved tplot variable, it should be written to the coordinate_system attribute in the CDF. data_att.coord_sys gets updated during cotrans operations, so it should supersede whatever might be in cdf.vatt.coordinate_system.

xandrd commented 1 year ago

Should spd_extract_tvar_metadata provide coordinate system as a metadata?

xandrd commented 1 year ago

Coordinate system support was added to tplot_add_cdf_structure.