Unidata / netcdf-cxx4

Official GitHub repository for netCDF-C++ libraries and utilities.
Other
124 stars 49 forks source link

getVar not correctly returning variable values #53

Open Chrismarsh opened 6 years ago

Chrismarsh commented 6 years ago

I'm seeing some very odd behaviour with the getVar method. Specifically, I'm seeing it return what looks to be uninitialised data, 0s, and repeated data when reading back an entire record's grid.

I'm seeing this behaviour across the following machines: Machine details: Fedora 27 -- netcdf-cxx4-devel 4.3.0-3 (from dnf); netcdf-4.4.1.1-6 (from dnf) MacOSX High Sierra-- netcdf-cxx4 (built from git); netcdf-4.5.0 (via homebrew) Ubuntu 16.04 -- netcdf (built from src, 4.5.1.-dev); netcdf-cxx4 (latest git); hdf5 5.1.10.1 (built from src) Building with gcc and clang.

Test program is as follows:

#include <iostream>
#include <netcdf>
#include <string>

int main()
{

    netCDF::NcFile data;
    data.open("small_nc.nc", netCDF::NcFile::read);

     auto dims = data.getDims();
    for(auto itr : dims)
    {
        std::cout << itr.first << "\t"<<itr.second.getSize()<< std::endl;
    }

    const size_t datetime_length = 6;
    const int NLAT = 4;
    const int NLON = 4;

    double lat[NLAT][NLON];
    double lon[NLAT][NLON];

    // read 1 record
    std::vector<size_t> startp, countp;
    startp.push_back(0);
    startp.push_back(0);
    startp.push_back(0);

    countp.push_back(1);
    countp.push_back(NLAT);
    countp.push_back(NLON);

    netCDF::NcVar latVar, lonVar;
    latVar = data.getVar("gridlat_0");
    if(latVar.isNull()) return -1;

    lonVar = data.getVar("gridlon_0");
    if(lonVar.isNull()) return -1;

    latVar.getVar(startp,countp, lat);
    lonVar.getVar(startp,countp, lon);

    for(int y = 0; y < NLAT; y++)
    {
        for(int x = 0; x < NLON; x++ )
        {
            std::cout << "array -- lat=" << lat[y][x] << "\tlon=" << lon[y][x] << std::endl;

            std::vector<size_t> startp_2, countp_2;
            startp_2.push_back(0);
            startp_2.push_back(y);
            startp_2.push_back(x);

            countp_2.push_back(1);
            countp_2.push_back(1);
            countp_2.push_back(1);

            double dlat,dlon;
            latVar.getVar(startp_2,countp_2, &dlat);
            lonVar.getVar(startp_2,countp_2, &dlon);

            std::cout << "index -- lat=" << dlat << "\tlon=" << dlon << std::endl;
        }
    }
    return 0;
}

I get the following output

datetime    6
xgrid_0 4
ygrid_0 4
array -- lat=8.06696e+10    lon=-6.29051e+14
index -- lat=44.6896    lon=-129.906
array -- lat=8.0874e+10 lon=-6.27106e+14
index -- lat=44.6896    lon=-129.906
array -- lat=6.9101e-310    lon=0
index -- lat=44.6896    lon=-129.906
array -- lat=6.9101e-310    lon=0
index -- lat=44.6896    lon=-129.906
array -- lat=0  lon=0
index -- lat=44.6956    lon=-129.879
array -- lat=0  lon=0
index -- lat=44.6956    lon=-129.879
array -- lat=9.88131e-324   lon=0
index -- lat=44.6956    lon=-129.879
array -- lat=0  lon=0
index -- lat=44.6956    lon=-129.879
array -- lat=0  lon=0
index -- lat=44.7015    lon=-129.851
array -- lat=0  lon=0
index -- lat=44.7015    lon=-129.851
array -- lat=0  lon=6.91692e-323
index -- lat=44.7015    lon=-129.851
array -- lat=0  lon=6.91692e-323
index -- lat=44.7015    lon=-129.851
array -- lat=3.45846e-323   lon=0
index -- lat=44.7075    lon=-129.823
array -- lat=0  lon=0
index -- lat=44.7075    lon=-129.823
array -- lat=0  lon=-nan
index -- lat=44.7075    lon=-129.823
array -- lat=0  lon=-nan
index -- lat=44.7075    lon=-129.823

As you can see, the array reads are a mess, where the single value read appears to be correct. However, on a much larger and production ncfile, I get output like the following:

[index -- lat=45.732    lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944
array -- lat=0  lon=0
index -- lat=45.732 lon=-123.944

where the array calls all return 0, and the individual calls end up returning duplicate data.

I've attached the files for this mwe below. It contains the tests data I've been using. mwe.zip

Chrismarsh commented 6 years ago

Follow up: my lat long grid does not, of course, have a time dimension. So reading individual x,y values like

            // startp_2.push_back(0);
            startp_2.push_back(y);
            startp_2.push_back(x);

            // countp_2.push_back(1);
            countp_2.push_back(1);
            countp_2.push_back(1);

            double dlat,dlon;
            latVar.getVar(startp_2,countp_2, &dlat);
            lonVar.getVar(startp_2,countp_2, &dlon);

Appears to work as expected. However, reading the entire grid as:

    // startp.push_back(0);
    startp.push_back(0);
    startp.push_back(0);

    // countp.push_back(1);
    countp.push_back(NLAT);
    countp.push_back(NLON);

    netCDF::NcVar latVar, lonVar;
    latVar = data.getVar("gridlat_0");
    if(latVar.isNull()) return -1;

    lonVar = data.getVar("gridlon_0");
    if(lonVar.isNull()) return -1;

does not work and still results in 0,0 values