Error in reading ODV file

[stefano-guicciardi]

Short summary of problem

When I try ro read a ODV file I get the following puzzling message

dati <- read.ctd("Prova.txt", type="ODV")

Error in read.ctd("Prova.txt", type = "ODV") : type must be SBE19, WOCE, ODF, ODV, or ITP, not ODV

How urgent is this?

I can wait.

Output from sessionInfo()

sessionInfo() R version 4.0.0 (2020-04-24) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale: [1] LC_COLLATE=Italian_Italy.1252 LC_CTYPE=Italian_Italy.1252 LC_MONETARY=Italian_Italy.1252 [4] LC_NUMERIC=C LC_TIME=Italian_Italy.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] ncdf4_1.17 oce_1.2-0 gsw_1.0-5 testthat_2.3.2 dplyr_0.8.5

loaded via a namespace (and not attached): [1] Rcpp_1.0.4.6 crayon_1.3.4 assertthat_0.2.1 R6_2.4.1 lifecycle_0.2.0 [6] magrittr_1.5 pillar_1.4.4 rlang_0.4.6 rstudioapi_0.11 vctrs_0.3.0
[11] ellipsis_0.3.0 tools_4.0.0 glue_1.4.0 purrr_0.3.4 compiler_4.0.0
[16] pkgconfig_2.0.3 tidyselect_1.1.0 tibble_3.0.1

[dankelley]

Can you email me (kelley.dan@gmail.com) the file you are trying to read?

[stefano-guicciardi]

Please find enclosed a typical ODV file.

Best regards,

Stefano Guicciardi

­­­­­____

Stefano Guicciardi

CNR-IRBIM

Largo Fiera della Pesca, 1

60125 Ancona

ITALY

Tel: +39-071-2078850

Fax: +39-071-55313

Mail: stefano.guicciardi@cnr.it

Skype: stefano.guicciardi58

Da: Dan Kelley notifications@github.com Inviato: venerdì 15 maggio 2020 12:19 A: dankelley/oce oce@noreply.github.com Cc: stefano-guicciardi stefano.guicciardi@an.ismar.cnr.it; Author author@noreply.github.com Oggetto: Re: [dankelley/oce] Error in reading ODV file (#1696)

Can you email me (kelley.dan@gmail.com mailto:kelley.dan@gmail.com ) the file you are trying to read?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dankelley/oce/issues/1696#issuecomment-629155861 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AD4XXAO7B6A4SR2AWU6LAZDRRUJLBANCNFSM4NBOKO2Q .

// //SDN_parameter_mapping //SDN:LOCAL:DEPHPR01SDN:P06::ULAA //SDN:LOCAL:PRESPR01SDN:P06::UPDB //SDN:LOCAL:OXYSSC01SDN:P06::UPCT //SDN:LOCAL:PHOSAADZSDN:P06::UPOX //SDN:LOCAL:ODSDM017SDN:P06::UPOX // Cruise Station Type yyyy-mm-ddThh:mm:ss.sss Longitude [degrees_east] Latitude [degrees_north] LOCAL_CDI_ID EDMO_code Bot. Depth [m] DEPHPR01 [m] QV:SEADATANET PRESPR01 [decibar=10000 pascals] QV:SEADATANET OXYSSC01 [%] QV:SEADATANET PHOSAADZ [millimole/m3] QV:SEADATANET ODSDM017 [millimole/m3] QV:SEADATANET Adricosm-EXT1 43 B 2005-11-06T12:15:00.000 +018.760000 +42.091000 Adricosm-EXT1_00043_H09 144 0.0 1.0 1 1.0 1 88.22410 1 0.06 1 1.70 1 9.4 1 9.5 1 0.35970 3 0.02 1 2.30 1 29.3 1 29.5 1 0.36500 3 0.02 1 1.19 1 68.9 1 69.5 1 0.35970 3 0.02 1 1.21 1 92.7 1 93.5 1 0.45700 3 0.04 1 1.76 1

[dankelley]

Please email the file, instead of putting it in a github comment. Otherwise I cannot know all the characters in the file. Use the address already provided.

[dankelley]

Hello. The "develop" branch of oce now has limited support for this file type (which is type="odv", not the upper-case version you were trying).

I call the support limited because it only recognizes a few column names. This is because ODV has a very large set of possible variable names, and no easy way to discover them. If you clone the repo and look in tests/testthat/local_data/odv (or look at https://github.com/dankelley/oce/tree/develop/tests/testthat/local_data/odv) you will see that we now have some test files. Although read.ctd.odv() reads all of these files, it can decode only a subset of column names. Try visiting that directory and running

for (file in list.files(path="local_data/odv", pattern="*.txt", full.names=TRUE)) {
   print(file)
   d <- read.ctd(file, type="ODV")
   summary(d)
}

to see what I mean.

See especially the 4th test file. For this, read.ctd.odv() decodes some column names (e.g. ammonium, with original name "NH4 [umol/l]") but not others, such as those with original names "NO3andNO2_Filt_ColAA [umol/l] " and "NO3andNO2_Filt_ColMan [umol/l]". I spent quite a bit of time trying to find the meanings of these two columns, but without any luck. Therefore, they must be accessed with those original (and cumbersome) column names.

The code for read.ctd.odv() has not worked since a stub was written, 4 years ago. No user has asked for this function to be fixed. This indicates that the file format is not popular enough to justify much more development effort, and certainly not without a comprehensive document that lists all possible column names (ideally in a machine-readable way).

I hope this helps a little. I realize that it will not help much. My advice is to use a different format, since the ODV format is awkward, poorly documented, and seemingly variable from file to file. Oce works better with manufacturer's formats and also file formats used by international archiving agencies.

[dankelley]

Dear reporter,

Do you think this issue (as defined by its title) has been addressed? If so, please close it. If not, please add a comment explaining what remains to be done. Thanks!

PS. This is a standardized reply.