search

DaniSb170 / nctoolbox

Automatically exported from code.google.com/p/nctoolbox
0 stars 0 forks source link

problem with grid_interop on vertical coordinates #27

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
% OPeNDAP Data URL for a CF-Compliant curvilinear POM model dataset
url=['http://michigan.glin.net:8080/thredds/dodsC/',...
'glos/glcfs/michigan/ncasf_his3d_fc/',...
'Lake_Michigan_Forecast_Collection_3D_best.ncd']
nc = ncgeodataset(url);
nc.variables
tvar = nc.geovariable('temp')
class(tvar)
size(tvar)
t= tvar.data(end, 1, 1:2:end, 1:2:end);  % works okay
tg1= tvar.grid(end, 1, 1:2:end, 1:2:end);  % works okay

tg2 = tvar.grid_interop(end, 1, 1:2:end, 1:2:end);  % bombs

Error using ncgeovariable/grid_interop (line 180)
There is a problem applying the vertical coordinate tranform and subsetting the 
resuting
values.

Error in ncgeovariable/subsref (line 637)
                                    sref = obj.grid_interop(first, last, stride);

Original issue reported on code.google.com by rsignell on 8 Nov 2011 at 4:08

GoogleCodeExporter commented 8 years ago 
Issue 26 has been merged into this issue.

Original comment by crosb...@gmail.com on 8 Nov 2011 at 4:19

GoogleCodeExporter commented 8 years ago 
Does this work in the old njTBX?

Original comment by crosb...@gmail.com on 17 Nov 2011 at 4:53

GoogleCodeExporter commented 8 years ago 
{{{
subgrid = grid.getVerticalTransform();
array = subgrid.getCoordinateArray(q-1); % Issue 27 is failing here...
}}}

Original comment by crosb...@gmail.com on 17 Nov 2011 at 4:58

GoogleCodeExporter commented 8 years ago 
Hi Brian, I am getting the following traceback/error on the above:

No method 'getCoordinateArray' with matching signature found for class 
'ucar.unidata.geoloc.vertical.OceanSG1'.

It's weird because  OceanSG1 does in fact have getCoordinateArray. On some 
datasets this fails on the first index/loop but on others 
(http://geoport.whoi.edu/thredds/dodsC/coawst_2_2/fmrc/coawst_2_2_best.ncd) it 
fails on the 4th or 5th loop. I don't know too much about java, am I 
implementing something wrong? Could I be blowing the heap, but not getting the 
error?

Alex

Original comment by crosb...@gmail.com on 3 Jan 2012 at 3:19

GoogleCodeExporter commented 8 years ago 
Hi Alex, I'll take a look either later today or tomorrow.

Original comment by bschlin...@gmail.com on 3 Jan 2012 at 4:59

GoogleCodeExporter commented 8 years ago 
Thanks, I did some profiling with jconsole, and I don't think its a memory issue

Original comment by crosb...@gmail.com on 3 Jan 2012 at 5:34

GoogleCodeExporter commented 8 years ago 
Also it appears to happening on requests that span multiple time indices

Original comment by crosb...@gmail.com on 4 Jan 2012 at 8:48

GoogleCodeExporter commented 8 years ago 
I get the following stacktrace:

opendap.dap.DataReadException: Inconsistent array length read: 1165128303 != 
1914731274
    at opendap.dap.DVector.deserialize(DVector.java:249)
    at opendap.dap.DataDDS.readData(DataDDS.java:117)
    at opendap.dap.DConnect2$DataDDSCommand.process(DConnect2.java:793)
    at opendap.dap.DConnect2.openConnection(DConnect2.java:313)
    at opendap.dap.DConnect2.getData(DConnect2.java:775)
    at opendap.dap.DConnect2.getData(DConnect2.java:1057)
    at ucar.nc2.dods.DODSNetcdfFile.readDataDDSfromServer(DODSNetcdfFile.java:1179)
    at ucar.nc2.dods.DODSNetcdfFile.readData(DODSNetcdfFile.java:1344)
    at ucar.nc2.Variable.reallyRead(Variable.java:860)
    at ucar.nc2.Variable._read(Variable.java:846)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:546)
    at ucar.nc2.dataset.VariableDS._read(VariableDS.java:526)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.Variable.read(Variable.java:604)
    at ucar.unidata.geoloc.vertical.VerticalTransformImpl.readArray(VerticalTransformImpl.java:130)
    at ucar.unidata.geoloc.vertical.OceanSigma.getCoordinateArray(OceanSigma.java:103)
opendap.dap.DAP2Exception: opendap.dap.DataReadException: Inconsistent array 
length read: 1165128303 != 1914731274
    at opendap.dap.DConnect2.openConnection(DConnect2.java:317)
    at opendap.dap.DConnect2.getData(DConnect2.java:775)
    at opendap.dap.DConnect2.getData(DConnect2.java:1057)
    at ucar.nc2.dods.DODSNetcdfFile.readDataDDSfromServer(DODSNetcdfFile.java:1179)
    at ucar.nc2.dods.DODSNetcdfFile.readData(DODSNetcdfFile.java:1344)
    at ucar.nc2.Variable.reallyRead(Variable.java:860)
    at ucar.nc2.Variable._read(Variable.java:846)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:546)
    at ucar.nc2.dataset.VariableDS._read(VariableDS.java:526)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.Variable.read(Variable.java:604)
    at ucar.unidata.geoloc.vertical.VerticalTransformImpl.readArray(VerticalTransformImpl.java:130)
    at ucar.unidata.geoloc.vertical.OceanSigma.getCoordinateArray(OceanSigma.java:103)
Caused by: opendap.dap.DataReadException: Inconsistent array length read: 
1165128303 != 1914731274
    at opendap.dap.DVector.deserialize(DVector.java:249)
    at opendap.dap.DataDDS.readData(DataDDS.java:117)
    at opendap.dap.DConnect2$DataDDSCommand.process(DConnect2.java:793)
    at opendap.dap.DConnect2.openConnection(DConnect2.java:313)
    ... 13 more

The corresponding Matlab errors are:
Error using ncgeovariable/grid_interop (line 213)
No method 'getCoordinateArray' with matching signature found for class 
'ucar.unidata.geoloc.vertical.OceanSigma'.

Error in ncgeovariable/subsref (line 1047)
                                    sref = obj.grid_interop(first, last, stride);

Original comment by bschlin...@gmail.com on 4 Jan 2012 at 11:16

GoogleCodeExporter commented 8 years ago 
Did a check at the command line for the method getCoordinateArray. The method 
does exist:

>> import ucar.unidata.geoloc.vertical.OceanSigma
>> methods('OceanSigma')

Methods for class ucar.unidata.geoloc.vertical.OceanSigma:

OceanSigma          getCoordinateArray  isTimeDependent     subset              
equals              getUnitString       notify              toString            
getClass            hashCode            notifyAll           wait  

Looks like the error "No method 'getCoordinateArray' with matching signature 
found for class 'ucar.unidata.geoloc.vertical.OceanSigma'." is a red herring. 
Might be a bug in the opendap libraries. I don't have time right now to dive 
into it though.

Original comment by bschlin...@gmail.com on 4 Jan 2012 at 11:21

GoogleCodeExporter commented 8 years ago 
Stepping through with a debugger:

Line 191: array = subgrid.getCoordinateArray(q-1);
throws: ucar.ma2.InvalidRangeException: i must be < length
    at ucar.ma2.Range.element(Range.java:381)
    at ucar.unidata.geoloc.vertical.VerticalTransformSubset.getCoordinateArray(VerticalTransformSubset.java:81)

Line 197:  array = subgrid.getCoordinateArray(q-1);
Values: q = 2972
            subgrid = ucar.unidata.geoloc.vertical.OceanSigma@73d84d77 (An instance of OceanSigma, different than subgrid at line 191)
throws: java.io.IOException: opendap.dap.DataReadException: Inconsistent array 
length read: 1165128303 != 1914731274
    at ucar.nc2.dods.DODSNetcdfFile.readData(DODSNetcdfFile.java:1351)
    at ucar.nc2.Variable.reallyRead(Variable.java:860)
    at ucar.nc2.Variable._read(Variable.java:846)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:546)
    at ucar.nc2.dataset.VariableDS._read(VariableDS.java:526)
    at ucar.nc2.Variable.read(Variable.java:658)
    at ucar.nc2.Variable.read(Variable.java:604)
    at ucar.unidata.geoloc.vertical.VerticalTransformImpl.readArray(VerticalTransformImpl.java:130)
    at ucar.unidata.geoloc.vertical.OceanSigma.getCoordinateArray(OceanSigma.java:103)

line 205: array = subgrid.getCoordinateArray();
Values: subgrid = ucar.unidata.geoloc.vertical.OceanSigma@73d84d77 (Same 
instance as line 197)
throws: MATLAB:UndefinedFunction - No method 'getCoordinateArray' with matching 
signature found for class 'ucar.unidata.geoloc.vertical.OceanSigma'.

So there's 2 things going on here:
1) We're getting an error from the opendap libraries at line 197. It's possible 
it's an opendap bug but it could easily be a incorrect index. So this needs to 
be double checked.
2) getCoordinateArray at line 197 requires an integer argument. It looks like 
the int value is ignored if: subgrid.isTimeDependant() == 0. In which case you 
can use any value as the argument, e.g.  array = subgrid.getCoordinateArray(0). 
See 
http://www.unidata.ucar.edu/software/netcdf-java/v4.2/javadoc/ucar/unidata/geolo
c/vertical/OceanSigma.html

Original comment by bschlin...@gmail.com on 5 Jan 2012 at 12:57

GoogleCodeExporter commented 8 years ago 
I think I am always providing an integer argument, but maybe I have to convert 
to integer explicitly?

Original comment by crosb...@gmail.com on 5 Jan 2012 at 2:03

GoogleCodeExporter commented 8 years ago 
The offending line was 

205: array = subgrid.getCoordinateArray();

I changed it to 

array = subgrid.getCoordinateArray(0);

No int conversion is needed, you were just missing an argument.

Original comment by bschlin...@gmail.com on 5 Jan 2012 at 4:06

GoogleCodeExporter commented 8 years ago 
Ah yeah, i see that. I didn't know anything ever actually hit that catch.

Original comment by crosb...@gmail.com on 5 Jan 2012 at 7:46

GoogleCodeExporter commented 8 years ago 
This problem was fixed but there is still the issue of the java being very slow 
on some sigma coordinate systems

Original comment by crosb...@gmail.com on 21 Jun 2012 at 4:07