IOS Conversion - moored currentmeter data issues

guanlu129 commented 4 months ago

For IOS moored Current meter data, I’ve compared the netCDF file generated by ODPY and by the old ios_data_tranform, and would like to suggest a few items for discussion and potential modification:

[ ] Varaible record_number is set as float in moored CTD file. The variable type should be consistent in moored CTD and moored current meter files. To do: change the record number from int to float.
[x] Adding long_name and standard_name to variable time as shown below: double time(time) ; time:_FillValue = NaN ; time:long_name = "time" ; time:standard_name = "time" ; time:units = "seconds since 1970-01-01 00:00:00+0000" ;
[ ] Missing variable instrument_depth in the ODPY version: float instrument_depth ; instrument_depth:_FillValue = NaNf ; instrument_depth:long_name = "Instrument Depth" ; instrument_depth:standard_name = "instrument_depth" ; instrument_depth:units = "m" ;

[ ] Variable names in the ODPY version are not BODC standard names. In ODPY version old version standard name in the old version

ios_name	BODC	standard_name
Speed\:East	LCEWEL01	"eastward_sea_water_velocity"
Speed\:North	LCNSEL01	"northward_sea_water_velocity"
Speed\:Up	LRZASP01	"upward_sea_water_velocity"
Amplitude\:Beam1	ISCMBMA1	"signal_intensity_frommultibeam.....”
Amplitude\:Beam2	ISCMBMA2	"signal_intensity_frommultibeam.....”
Amplitude\:Beam3	ISCMBMA3	"signal_intensity_frommultibeam.....”
Speed\:Sound\:1	SVELCV01	"speed_of_sound_in_sea_water"
Heading	HEADCM01	"platform_orientation"
Pitch	PTCHEI01	"platform_pitch"
Roll	ROLLEI01	"platform_roll"
Speed\:Current	LCSAEL01	"sea_water_speed"

Question: should I add these variables to the dfo_ios_vocabulary.csv

[ ] There are a few metadata under data section in the old version:

filename = "chat3_20220722_20230802_0075m_L1.cur" ;
country = "Canada" ;
deployment_mission_id = "2023-069" ;
event_number = "4" ;
profile = "2023-069-0004" ;
scientist = "Hannah C." ;
project = "BC-shelf 7" ;
agency = "IOS, Ocean Sciences Division, Sidney, BC" ;
platform = "John P. Tully" ;
instrument_type = "Aquadopp Current Meter" ;
instrument_model = "AQUADOPP" ;
instrument_serial_number = "AQUADOPP-17257" ;
instrument_depth = 75 ;
latitude = 54.52715 ;
longitude = -130.5415 ;
geographic_area = "Hecate-Strait " ;

Question: are these necessary?

JessyBarrette commented 4 months ago

Few comments from above

Varaible record_number is set as float in moored CTD file. The variable type should be consistent in moored CTD and moored current meter files. To do: change the record number from int to float.

the parser relies on the original file CHANNEL DETAIL to define the variable data type. As an example from A1_20130707_20140626_0185m.ctd:

    $TABLE: CHANNELS
    ! No Name            Units        Minimum        Maximum
    !--- --------------- ------------ -------------- --------------
       1 Record_Number   n/a          1              16979
       2 Date            YYYY/MM/DD   0              354
       3 Time            HH:MM:SS     0.2314815E-04  0.9791898
       4 Temperature     'deg C'      6.1785         8.1052
       5 Pressure        decibar      180.469        184.555
       6 Salinity        PSS-78       33.6531        33.9793
       7 Speed:Sound     m/s          1477.139       1484.307
    $END

    $TABLE: CHANNEL DETAIL
    ! No  Pad   Start  Width  Format      Type  Decimal_Places
    !---  ----  -----  -----  ----------  ----  --------------
       1  -99   ' '        8  F           R4      1
       2  -99   ' '    ' '    YYYY/MM/DD  D     ' '
       3  -99   ' '    ' '    HH:MM:SS    T     ' '
       4  ' '   ' '        9  f           R4      4
       5  ' '   ' '        9  f           R4      3
       6  ' '   ' '        9  f           R4      4
       7  ' '   ' '        9  f           R4      3
    $END

The difference you observed is likely related to the a different format defined within each source files. We could potentially start mapping each individual variables data type via the vocabulary file but I could see this been a lot of work for not much reward. ERDDAP and other softwares are able to handle a mix of float/int back into int as long as those floats do not have decimals.

Variable names in the ODPY version are not BODC standard names. In ODPY version old version standard name in the old version

I'm surprised that those variables were added yet in the vocabulary. You can certainly add them to the vocabulary or I can do. It can be a good practice for you to familiarize yourself with all the git world 😄

JessyBarrette commented 4 months ago

@guanlu129 you can either make changes on your own computer by pulling the branch fix-74-dfo-ios-update and pushing them back.

or by making the changes directly via the GitHub interface here: https://github.com/cioos-siooc/ocean-data-parser/blob/fix-74-dfo-ios-update/ocean_data_parser/vocabularies/dfo_ios_vocabulary.csv

The second is probably the simplest.

cioos-siooc / ocean-data-parser

IOS Conversion - moored currentmeter data issues #82