search

ArgoCanada / argoFloats

Tools for analyzing collections of oceanographic Argo floats
https://argocanada.github.io/argoFloats/index.html
17 stars 7 forks source link

Date missing from readProfiles() output #593

Closed MarkTaylorNOC closed 1 year ago

MarkTaylorNOC commented 1 year ago

Recently, I have not been able to read in the date/time variable of profiles using readProfiles() (although I have many times in the past). When using the example code from the package help file, the argoFloats object reports all variables (latitude, longitude, temperature, etc...) except the date. 

 library(argoFloats)
 ai <- getIndex("synthetic")
 sub <- subset(ai, 1:2) # To subset for profiles
 profiles <- getProfiles(sub)
 argos <- readProfiles(profiles)

I've emailed the netcdf of the downloaded file (SD1900722_003.nc) to Kelley. The resulting output looks normal other than the absence of the time variable.

Screenshot (82)

I've tried uninstalling and updating the package, as well as using older versions, however this hasn't made a difference. This only happens on my work PC (Windows), and identical code works fine on my laptop (also Windows).

I'm currently using the following versions: R version 4.3.0 (2023-04-21 ucrt) argoFloats 1.0.6, oce 1.8-0 ncdf4 1.21

Here are some details of my machine: platform x86_64-w64-mingw32
arch x86_64
os mingw32

dankelley commented 1 year ago

Hi. Using the nc file you sent, I get as below (using the very handy reprex::reprex() function). Can you try this and report what you get on the problematic machine?

I want to isolate the problem. Note that argoFloats uses oce to try to read the file, and that oce in turn uses ncdf to read the raw contents. So, my questions are

  1. does this reprex illlustrate the problem? (If so, this issue ought to be on the github.com/dankelley/oce issue list.)

  2. what are you expecting to get? (Please express this in terms of code, not snapshots.)

``` r f <- "~/Downloads/SD1900722_003.nc" # 1. test with oce library(oce) #> Loading required package: gsw a <- read.argo(f) str(a@data) # note that MTIME is not there #> List of 14 #> $ latitude : num -40.5 #> $ longitude : num 73.3 #> $ pressure : num [1:71, 1] 6.5 10.5 19.8 29.7 40.4 ... #> $ pressureAdjusted : num [1:71, 1] 6.7 10.7 20 29.9 40.6 ... #> $ pressureAdjustedError : num [1:71, 1] 2.4 2.4 2.4 2.4 2.4 ... #> $ temperature : num [1:71, 1] 13.2 13.2 13.1 13 12.9 ... #> $ temperatureAdjusted : num [1:71, 1] 13.2 13.2 13.1 13 12.9 ... #> $ temperatureAdjustedError: num [1:71, 1] 0.002 0.002 0.002 0.002 0.002 ... #> $ salinity : num [1:71, 1] 35.1 35.1 35.1 35.1 35.1 ... #> $ salinityAdjusted : num [1:71, 1] 35.1 35.1 35.1 35.1 35.1 ... #> $ salinityAdjustedError : num [1:71, 1] 0.01 0.01 0.01 0.01 0.01 ... #> $ oxygen : num [1:71, 1] 232 233 235 236 234 ... #> $ oxygenAdjusted : num [1:71, 1] 261 261 263 264 263 ... #> $ oxygenAdjustedError : num [1:71, 1] 1.59 1.59 1.59 1.59 1.59 ... str(a@metadata) # note that MTIME is not there #> List of 51 #> $ units :List of 14 #> ..$ latitude :List of 2 #> .. ..$ unit : expression(degree * N) #> .. ..$ scale: chr "" #> ..$ longitude :List of 2 #> .. ..$ unit : expression(degree * E) #> .. ..$ scale: chr "" #> ..$ oxygen :List of 2 #> .. ..$ unit : expression(mu * mol/kg) #> .. ..$ scale: chr "" #> ..$ oxygenAdjusted :List of 2 #> .. ..$ unit : expression(mu * mol/kg) #> .. ..$ scale: chr "" #> ..$ oxygenAdjustedError :List of 2 #> .. ..$ unit : expression(mu * mol/kg) #> .. ..$ scale: chr "" #> ..$ temperature :List of 2 #> .. ..$ unit : expression(degree * C) #> .. ..$ scale: chr "ITS-90" #> ..$ temperatureAdjusted :List of 2 #> .. ..$ unit : expression(degree * C) #> .. ..$ scale: chr "ITS-90" #> ..$ temperatureAdjustedError:List of 2 #> .. ..$ unit : expression(degree * C) #> .. ..$ scale: chr "ITS-90" #> ..$ salinity :List of 2 #> .. ..$ unit : expression() #> .. ..$ scale: chr "PSS-78" #> ..$ salinityAdjusted :List of 2 #> .. ..$ unit : expression() #> .. ..$ scale: chr "PSS-78" #> ..$ salinityAdjustedError :List of 2 #> .. ..$ unit : expression() #> .. ..$ scale: chr "PSS-78" #> ..$ pressure :List of 2 #> .. ..$ unit : expression(dbar) #> .. ..$ scale: chr "" #> ..$ pressureAdjusted :List of 2 #> .. ..$ unit : expression(dbar) #> .. ..$ scale: chr "" #> ..$ pressureAdjustedError :List of 2 #> .. ..$ unit : expression(dbar) #> .. ..$ scale: chr "" #> $ flags :List of 8 #> ..$ pressure : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ pressureAdjusted : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ temperature : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ temperatureAdjusted: num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ salinity : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ salinityAdjusted : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> ..$ oxygen : num [1:71, 1] 3 3 3 3 3 3 3 3 3 3 ... #> ..$ oxygenAdjusted : num [1:71, 1] 1 1 1 1 1 1 1 1 1 1 ... #> $ filename : chr "/Users/kelley/Downloads/SD1900722_003.nc" #> $ flagScheme :List of 3 #> ..$ name : chr "argo" #> ..$ mapping:List of 10 #> .. ..$ not_assessed : num 0 #> .. ..$ passed_all_tests: num 1 #> .. ..$ probably_good : num 2 #> .. ..$ probably_bad : num 3 #> .. ..$ bad : num 4 #> .. ..$ changed : num 5 #> .. ..$ not_used_6 : num 6 #> .. ..$ not_used_7 : num 7 #> .. ..$ estimated : num 8 #> .. ..$ missing : num 9 #> ..$ default: num [1:4] 0 3 4 9 #> $ title : chr "Argo float vertical profile" #> $ institution : chr "AOML" #> $ source : chr "Argo float" #> $ history : chr "2022-06-28T08:08:26Z creation (software version 1.13 (version 09.06.2022 for ARGO_simplified_profile))" #> $ references : chr "http://www.argodatamgt.org/Documentation" #> $ userManualVersion : chr "1.0" #> $ conventions : chr "Argo-3.1 CF-1.6" #> $ featureType : chr "trajectoryProfile" #> $ id : chr "1900722" #> $ projectName : chr "US ARGO PROJECT" #> $ PIName : chr "STEPHEN RISER" #> $ nParameters : int 4 #> $ nProfiles : int 1 #> $ stationParameters : chr [1:4, 1] "PRES" "TEMP" "PSAL" "DOXY" #> $ cycleNumber : int 3 #> $ direction : chr "A" #> $ dataCentre : chr "AO" #> $ firmwareVersion : chr "012606" #> $ WMOInstType : chr "846" #> $ juld : num 20768 #> $ time : POSIXct[1:1], format: "2006-11-11 10:12:21" #> $ juldQC : chr "1" #> $ juldLocation : num 20768 #> $ dataNamesOriginal :List of 14 #> ..$ latitude : chr "LATITUDE" #> ..$ longitude : chr "LONGITUDE" #> ..$ pressure : chr "PRES" #> ..$ pressureAdjusted : chr "PRES_ADJUSTED" #> ..$ pressureAdjustedError : chr "PRES_ADJUSTED_ERROR" #> ..$ temperature : chr "TEMP" #> ..$ temperatureAdjusted : chr "TEMP_ADJUSTED" #> ..$ temperatureAdjustedError: chr "TEMP_ADJUSTED_ERROR" #> ..$ salinity : chr "PSAL" #> ..$ salinityAdjusted : chr "PSAL_ADJUSTED" #> ..$ salinityAdjustedError : chr "PSAL_ADJUSTED_ERROR" #> ..$ oxygen : chr "DOXY" #> ..$ oxygenAdjusted : chr "DOXY_ADJUSTED" #> ..$ oxygenAdjustedError : chr "DOXY_ADJUSTED_ERROR" #> $ positionQC : chr "1" #> $ positioningSystem : chr "ARGOS" #> $ dataType : chr "Argo synthetic profile" #> $ formatVersion : chr "1.0" #> $ handbookVersion : chr "1.2" #> $ dateCreation : chr "20220628080826" #> $ dateUpdate : chr "20220628080826" #> $ parameterDataMode : chr "DDDD" #> $ platformType : chr "APEX" #> $ floatSerialNo : chr "2596" #> $ configMissionNumber : int 1 #> $ parameter : chr [1:4, 1, 1] "PRES" "TEMP" "PSAL" "DOXY" #> $ scientificCalibEquation : chr [1:4, 1, 1] "PRES_ADJUSTED = PRES - dP" "none" "PSAL_ADJUSTED = sw_salt( sw_cndr(PSAL,TEMP,PRES), TEMP, PRES_ADJUSTED ) + CTM" "DOXY_MOLAR (umol L-1) = {[(C4 + C5*TEMP) / (C6 + C7*BPHASE_DOXY)] - 1} / {C1 + C2*TEMP + C3*TEMP^2}; ADJUSTED_D"| __truncated__ #> $ scientificCalibCoefficient: chr [1:4, 1, 1] "dP =-0.2 dbar." "none" "CTM: alpha=0.0267, tau=18.6s, mean_ascent_rate = 0.09 dbar/s" "C1 = 0.0028485; C2 = 0.0028485; C3 = 1.5054e-06; C4 = 70.423; C5 = -0.10302; C6 = -9.7617; C7 = 1.179" #> $ scientificCalibComment : chr [1:4, 1, 1] "Pressures adjusted by using pressure offset at the sea surface. The quoted error is manufacturer specified accuracy in dbar." "The quoted error is manufacturer specified accuracy with respect to ITS-90 at time of laboratory calibration." "No significant salinity drift detected. Salinity adjusted for effects of pressure adjustment. Conductivity cell"| __truncated__ "DOXY_ADJUSTED computed using Stern-Volmer equation with coeffs refit from foil calibration data & WOD data as i"| __truncated__ #> $ scientificCalibDate : chr [1:4, 1, 1] "20120705140756" "20120705140756" "20120705140756" "20150822000000" #> $ profilePresQC : chr "A" #> $ profileTempQC : chr "A" #> $ profilePsalQC : chr "A" #> $ profileDoxyQC : chr "A" #> $ tempDpres : num [1:71] 0 0 0 0 0 0 0 0 0 0 ... #> $ psalDpres : num [1:71] 0 0 0 0 0 0 0 0 0 0 ... #> $ doxyDpres : num [1:71] 0 0 0 0 0 0 0 0 0 0 ... # 1. test with ncdf (which is what oce uses for some of the work) library(ncdf4) nc <- nc_open(f) sort(names(nc$var)) #> [1] "CONFIG_MISSION_NUMBER" "CYCLE_NUMBER" #> [3] "DATA_CENTRE" "DATA_TYPE" #> [5] "DATE_CREATION" "DATE_UPDATE" #> [7] "DIRECTION" "DOXY" #> [9] "DOXY_ADJUSTED" "DOXY_ADJUSTED_ERROR" #> [11] "DOXY_ADJUSTED_QC" "DOXY_dPRES" #> [13] "DOXY_QC" "FIRMWARE_VERSION" #> [15] "FLOAT_SERIAL_NO" "FORMAT_VERSION" #> [17] "HANDBOOK_VERSION" "JULD" #> [19] "JULD_LOCATION" "JULD_QC" #> [21] "LATITUDE" "LONGITUDE" #> [23] "PARAMETER" "PARAMETER_DATA_MODE" #> [25] "PI_NAME" "PLATFORM_NUMBER" #> [27] "PLATFORM_TYPE" "POSITION_QC" #> [29] "POSITIONING_SYSTEM" "PRES" #> [31] "PRES_ADJUSTED" "PRES_ADJUSTED_ERROR" #> [33] "PRES_ADJUSTED_QC" "PRES_QC" #> [35] "PROFILE_DOXY_QC" "PROFILE_PRES_QC" #> [37] "PROFILE_PSAL_QC" "PROFILE_TEMP_QC" #> [39] "PROJECT_NAME" "PSAL" #> [41] "PSAL_ADJUSTED" "PSAL_ADJUSTED_ERROR" #> [43] "PSAL_ADJUSTED_QC" "PSAL_dPRES" #> [45] "PSAL_QC" "REFERENCE_DATE_TIME" #> [47] "SCIENTIFIC_CALIB_COEFFICIENT" "SCIENTIFIC_CALIB_COMMENT" #> [49] "SCIENTIFIC_CALIB_DATE" "SCIENTIFIC_CALIB_EQUATION" #> [51] "STATION_PARAMETERS" "TEMP" #> [53] "TEMP_ADJUSTED" "TEMP_ADJUSTED_ERROR" #> [55] "TEMP_ADJUSTED_QC" "TEMP_dPRES" #> [57] "TEMP_QC" "WMO_INST_TYPE" ``` Created on 2023-07-11 with [reprex v2.0.2](https://reprex.tidyverse.org)
dankelley commented 1 year ago

I forgot to say: in the reprex from the previous comment, if I add a[["time"]] I see just a single time. That makes sense, because this file has no MTIME data (as revealed by the direct ncdf4 work).

So I think there is no issue. Time has just a single value because this profile does not have an MTIME field. But now I'm confused. Are you saying that some machines give time an others do not?

I am really quite sure this is an oce issue, not an argoFloats issue. So I'd like to shift the discussion there, unless you disagree, @MarkTaylorNOC. The reason is that I check on oce issues quite frequently, and am not even notified when argoFloats issues are reported (since argoFloats is in a group repo, not an individual repo).

MarkTaylorNOC commented 1 year ago

I'm happy with moving the issue to oce. Can the issue be transferred on Github or should I copy and paste the issue onto the oce page?

dankelley commented 1 year ago

I think the easiest is to copy/paste. Thanks for the help.

PS. an advantage of being on oce is that it will attract more attention there, which might speed things up a bit (with travel and other work assignments, I'm the only one available for argoFloats for a month or so).

MarkTaylorNOC commented 1 year ago

Okay, thanks. I will set up a new issue on oce.

dankelley commented 1 year ago

Discussion has been moved over to https://github.com/dankelley/oce/issues/2122 so I'm closing the present issue.