EnableCF=false cannot retrieve DODS object.

[hyoklee]

This bug is found with PyDAP client. Hyrax version is http://54.174.38.12:8888/opendap/version

1. Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
2. Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
3. Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5.dods file with text editor, you will see:

Error {
    code = 1005;
    message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data";
};

PyDap will throw an error accordingly when it tries to test EnableCF=false.

[ndp-opendap]

The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.

I can replicate the issue with curl:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

It’s quite interesting that the ASCII response Works:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999

As does the DDS response:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;

However, the DMR request fails (and with a VERY interesting error message):

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" <?xml version="1.0" encoding="ISO-8859-1"?>

1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org

On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com wrote:

Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html

Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m

Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:

Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

PyDap will throw an error accordingly when it tries to test EnableCF=false.

— Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

[ndp-opendap]

Joe,

Can you make this dataset available for download please?

http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5

Thanks,

Nathan

On Feb 3, 2016, at 3:50 PM, Nathan Potter ndp@opendap.org wrote:

The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.

I can replicate the issue with curl:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

It’s quite interesting that the ASCII response Works:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999

As does the DDS response:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;

However, the DMR request fails (and with a VERY interesting error message):

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" <?xml version="1.0" encoding="ISO-8859-1"?>

1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org

On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com wrote:

Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html

Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m

Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:

Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

PyDap will throw an error accordingly when it tries to test EnableCF=false.

— Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

[hyoklee]

@ndp-opendap You can download it using the following URL:

http://54.174.38.12/thredds/fileServer/testAll/hpd/GSSTF_NCEP.3.2008.01.01.he5

[kyang2014]

I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline

Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)

The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.

I can replicate the issue with curl:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

It’s quite interesting that the ASCII response Works:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999

As does the DDS response:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;

However, the DMR request fails (and with a VERY interesting error message):

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>

1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org>

On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:

Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html

Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m

Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:

Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

PyDap will throw an error accordingly when it tries to test EnableCF=false.

— Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275.

[ndp-opendap]

Kent,

How was the hdf5_handler configured to produce this DMR?

N

On Feb 4, 2016, at 7:22 AM, kyang2014 notifications@github.com wrote:

I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline

Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)

The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.

I can replicate the issue with curl:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

It’s quite interesting that the ASCII response Works:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999

As does the DDS response:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;

However, the DMR request fails (and with a VERY interesting error message):

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>

1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org>

On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:

Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html

Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m

Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:

Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

PyDap will throw an error accordingly when it tries to test EnableCF=false.

— Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275. — Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

[ndp-opendap]

Kent,

I set "H5.EnableCF=false" and then I was able to see the naming as in the baseline you referred to.

However I am not able to reproduce the issue, so this problem may be fixed in the current code??

I would appreciate it if you (or Joe) would please make the original data file that spawned the problem report available to me so that I can confirm.

http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5

Thanks,

Nathan

On Feb 4, 2016, at 10:17 AM, Nathan Potter ndp@opendap.org wrote:

Kent,

How was the hdf5_handler configured to produce this DMR?

N

On Feb 4, 2016, at 7:22 AM, kyang2014 notifications@github.com wrote:

I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline

Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)

The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.

I can replicate the issue with curl:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

It’s quite interesting that the ASCII response Works:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999

As does the DDS response:

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;

However, the DMR request fails (and with a VERY interesting error message):

curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>

1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org>

On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:

Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html

Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m

Press "Binary (DAP2) Object".

The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:

Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };

PyDap will throw an error accordingly when it tries to test EnableCF=false.

— Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275. — Reply to this email directly or view it on GitHub.

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317

[ndp-opendap]

I got the file:

http://54.174.38.12/thredds/fileServer/testAll/hpd/GSSTF_NCEP.3.2008.01.01.he5

And tried it on my dev system against the current master branch of everything and it works fine.

The Data request form retrieves the correct data using this URL: http://localhost:8080/opendap/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]

And using curl I can pipe that URL into getdap and I get the expected result. curl -s "http://localhost:8080/opendap/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" | getdap -M -s -

The fact that this is not tested in the hdf5_handler tests is a problem and it would be great if it where. The BES command that results in dereferencing that URL might look like:

<?xml version="1.0" encoding="UTF-8"?>
<bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="http-bio-8080-exec-7:28">
  <bes:setContext name="bes_timeout">40</bes:setContext>
  <bes:setContext name="xdap_accept">2.0</bes:setContext>
  <bes:setContext name="dap_explicit_containers">no</bes:setContext>
  <bes:setContext name="errors">xml</bes:setContext>
  <bes:setContext name="max_response_size">0</bes:setContext>
  <bes:setContainer name="catalogContainer" space="catalog">/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5</bes:setContainer>
  <bes:define name="d1" space="default">
    <bes:container name="catalogContainer">
      <bes:constraint>/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]</bes:constraint>
    </bes:container>
  </bes:define>
  <bes:get type="dods" definition="d1" />
</bes:request>

Modulo the value of the <setContainer> element: /data/hdf5/GSSTF_NCEP.3.2008.01.01.he5

In general I think a good thing to do would be to verify that a suspected bug is still a problem with the current code before creating a bug report because often, as in this case, the issue has been fixed.

[hyoklee]

I upgraded the server to Hyrax 1.13 and I still can't get DAP2 binary response.

http://54.174.38.12:8888/opendap/version

[ndp-opendap]

The version response doesn't look like you have completed the upgrade:

<Hyrax version="1.12.2" />
<OLFS version="1.14.1" />

What did you do? Did you get the release from the web page or did you get pieces from GitHub?

[hyoklee]

I got it from website and copied new olfs into webapps/. Let me try restarting the server and deleting old content/ directory.

[ndp-opendap]

You should read the installation instructions (and don't delete the content directory if you have modified it). Because the OLFS has changed significantly in the way that it locates it's configuration.

[ndp-opendap]

I just looked at http://54.174.38.12:8888/opendap/version and I see that it's now up-to-date in the version.

But now it appears from the structure of: http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds That the configuration parameter:

H5.EnableCF=true

Is that what you intend?

[hyoklee]

I'm sorry for the late response. I was in a meeting and you saw the response of from the default olfs.xml server configuration. After adjusting port in olfs.xml to talk to a different BES, I could get the desired output of hdf5_handler with H5.EnableCF=false. You can check the above URL again.

By the way, I could confirm that this bug is fixed with the latest OLFS. Thanks for fixing it in a new release. I can resume my Hyrax vs. THREDDS testing with EnableCF=true/false.

[ndp-opendap]

Thanks for the follow up Joe, I'm glad you are getting the same results I am now.

Question: Did you set the OLFS_CONFIG_DIR environment variable or did you create /etc/olfs to localize the OLFS configuration?

[hyoklee]

I just modified webapps/opendap/olfs.xml directly. It was interesting to see that content/opendap is not created any more.

[ndp-opendap]

What you did will work for awhile, but will not persist. If the war file is unpacked again or a new one installed all of your changes will be lost. Please consider reading: http://docs.opendap.org/index.php/Hyrax_-_OLFS_Configuration#OLFS_Configuration_Location

And use either the OLFS_CONFIG_DIR environment variable or the well known location /etc/olfs to store your localized configuration, otherwise you may find the server behaving unpredictably down the road.

These changes where made because most user (NASA especially) install Tomcat from YUM and the content directory strategy was broken for those systems.