search

dankelley / oce

R package for oceanographic processing
http://dankelley.github.io/oce/
GNU General Public License v3.0
145 stars 42 forks source link

local build-test error re CTD pressure in a local file #1534

Closed dankelley closed 5 years ago

dankelley commented 5 years ago

Below is the error. I encountered this just now, with the newly-installed R 3.6.0, but I think it relates to #1166, and I just didn't see it because I was building from another working directory and thus was not capturing this local file (not provided with oce on CRAN).

test_local_ctd.R:92: failure: autoconverts pressure in PSI to in dbar
d1[["pressure"]] not equal to d2[["pressure"]].
181/181 mismatches (average diff: 12)
[1] 1.48 - 0.704 == 0.776
[2] 1.67 - 0.794 == 0.877
[3] 2.05 - 0.975 == 1.077
[4] 2.24 - 1.066 == 1.177
[5] 2.62 - 1.247 == 1.377
[6] 2.67 - 1.270 == 1.402
[7] 3.00 - 1.429 == 1.576
[8] 3.20 - 1.519 == 1.677
[9] 3.48 - 1.655 == 1.827
...
dankelley commented 5 years ago

The following shows that the two pressures are in constant ratio.

d1 <- expect_warning(read.oce("local_data/ctd.cnv"), "this CNV file has temperature in the IPTS-68 scale")
p1 <- d1[['pressure']]
d2 <- expect_warning(read.oce("local_data/ctd_with_psi.cnv"), "created 'pressure' from 'pressurePSI'")
p2 <- d2[['pressure']]
lm(p2~p1)

output:

Call:
lm(formula = p2 ~ p1)

Coefficients:
(Intercept)           p1  
 -1.459e-08    4.754e-01
dankelley commented 5 years ago

The data (first 3 lines following the *END* line) are as follows.

d1 constructed from file local_data/ctd.cnv, which has 3rd column being pressure in dbar:

        130    129.000      1.480      1.468    14.2245    29.9210  0.000e+00
        131    130.000      1.671      1.657    14.2299    29.9205  0.000e+00
        132    131.000      2.052      2.036    14.2285    29.9206  0.000e+00

d2 constructed from file local_data/ctd_with_psi.cnv, which has 3rd column being pressure in PSI:

        130    129.000  1.020424      1.468    14.2245    29.9210  0.000e+00
        131    130.000  1.152114      1.657    14.2299    29.9205  0.000e+00
        132    131.000  1.414804      2.036    14.2285    29.9206  0.000e+00
dankelley commented 5 years ago

Hm, I'm lost here. These data files were supposedly created using seabird software from the same input file, C:\SEASOFT3\BASIN\BED0302.HEX. (I didn't create the files, and am going just from the headers in them.)

The numbers do not make sense. Take the first value. In local_data/ctd.cnv the pressure is stated as 1.480 dbar but in local_data/ctd_with_psi.cnv it is stated as 1.020424. But those values do not match. (Recall that 1atm is 10dbar, and also 14PSI ... the numerical value for pressure in PSI must be larger than that in dbar, not smaller.)

So, now I am questioning the local_data/ctd.cnv file. Since I don't have a windows machine, and I am pretty terrible at using either 'wine' or 'virtualbox' to try to run the windows software from SBE that creates .cnv files, I wonder where I even got this local-data/ctd_with_psi.cnv file. I see two possible sources: (a) it came from SBE software, in which case I worry that the software is giving incorrect results or (b) I faked the file, for a test of how oce could convert from PSI data files. In either case, what we have is a case of dividing by the appropriate scale factor, instead of multiplying by it. Here's the evidence:

> range(d1[['pressure']]/d2[['pressure']]* 0.6894757293168^2)
[1] 0.9999998 1.0000002

@richardsc if I can dig up the original hex file, could you try running it through SBE software to see if it's case (a) above? It may take me a while to try to find the file, but I think there's a small chance it's on my work machine, somewhere.

dankelley commented 5 years ago

The odd number of digits in the pressure field of local_data/ctd_with_psi.cnv makes me think that I had created that pressure column (and done so incorrectly).

I've had a pretty good look for the .hex file and nothing has turned up. I don't want oce to fail the build-tests, so I am going to edit local_data/ctd_with_psi.cnv manually, replacing with values in PSI. By the magic of git, we can always go back, if needed, or update the file if we can regenerate it with SBE software.

dankelley commented 5 years ago

Done in "develop" branch, as shown below. I will reopen this if I can find the .hex file, but for now, I'm probably 90% certain that this local_data/ctd_with_psi.cnv file must have been created by me, by hand, in error.

commit 6de084681b94c1899b290517a925e86e0614869b (HEAD -> develop, origin/develop, origin/HEAD) Author: dankelley kelley.dan@gmail.com Date: Sun Apr 28 09:49:06 2019 -0300

hand-edit ctd_with_psi.cnv for pressure

See discussion at

https://github.com/dankelley/oce/issues/1534