Closed hyoklee closed 8 years ago
The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.
I can replicate the issue with curl:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
It’s quite interesting that the ASCII response Works:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999
As does the DDS response:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;
However, the DMR request fails (and with a VERY interesting error message):
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" <?xml version="1.0" encoding="ISO-8859-1"?>
On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com wrote:
Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
Press "Binary (DAP2) Object".
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:
Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
PyDap will throw an error accordingly when it tries to test EnableCF=false.
— Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
Joe,
Can you make this dataset available for download please?
http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5
Thanks,
Nathan
On Feb 3, 2016, at 3:50 PM, Nathan Potter ndp@opendap.org wrote:
The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.
I can replicate the issue with curl:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
It’s quite interesting that the ASCII response Works:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999
As does the DDS response:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;
However, the DMR request fails (and with a VERY interesting error message):
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" <?xml version="1.0" encoding="ISO-8859-1"?>
1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com wrote:
Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
Press "Binary (DAP2) Object".
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:
Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
PyDap will throw an error accordingly when it tries to test EnableCF=false.
— Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
@ndp-opendap You can download it using the following URL:
http://54.174.38.12/thredds/fileServer/testAll/hpd/GSSTF_NCEP.3.2008.01.01.he5
I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline
Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)
The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.
I can replicate the issue with curl:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
It’s quite interesting that the ASCII response Works:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999
As does the DDS response:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;
However, the DMR request fails (and with a VERY interesting error message):
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>
On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:
Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
Press "Binary (DAP2) Object".
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:
Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
PyDap will throw an error accordingly when it tries to test EnableCF=false.
— Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275.
Kent,
How was the hdf5_handler configured to produce this DMR?
N
On Feb 4, 2016, at 7:22 AM, kyang2014 notifications@github.com wrote:
I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline
Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)
The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.
I can replicate the issue with curl:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
It’s quite interesting that the ASCII response Works:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999
As does the DDS response:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;
However, the DMR request fails (and with a VERY interesting error message):
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>
1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org >On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:
Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
Press "Binary (DAP2) Object".
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:
Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
PyDap will throw an error accordingly when it tries to test EnableCF=false.
— Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275. — Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
Kent,
I set "H5.EnableCF=false" and then I was able to see the naming as in the baseline you referred to.
However I am not able to reproduce the issue, so this problem may be fixed in the current code??
I would appreciate it if you (or Joe) would please make the original data file that spawned the problem report available to me so that I can confirm.
http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5
Thanks,
Nathan
On Feb 4, 2016, at 10:17 AM, Nathan Potter ndp@opendap.org wrote:
Kent,
How was the hdf5_handler configured to produce this DMR?
N
On Feb 4, 2016, at 7:22 AM, kyang2014 notifications@github.com wrote:
I am pretty sure this is due to the space in the variable name. Some time ago P.O. DAAC reported a similar issue. The DMR output is fine with besstandalone at the HDF5 handler. Check https://github.com/OPENDAP/hdf5_handler/blob/master/bes-testsuite/h5.nasa.default.dap4/GSSTF.2b.2008.01.01.he5.dmr.bescmd.baseline
Kent From: Nathan Potter [mailto:notifications@github.com] Sent: Wednesday, February 03, 2016 5:50 PM To: OPENDAP/hdf5_handler Cc: Kent Yang Subject: Re: [hdf5_handler] EnableCF=false cannot retrieve DODS object. (#4)
The DataRequest Form appears to be generating the correct request URLs, but the DAP2 response not working. Th problem seems to be the space character in the variable name, which the Form is correctly encoding as %20.
I can replicate the issue with curl:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
It’s quite interesting that the ASCII response Works:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.ascii?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset: GSSTF_NCEP.3.2008.01.01.he5 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[1], 0.7005, -999, 0.83313, 0.837128, 0.444244, -999, -999, -999, -1.54453, -999, -0.528442, -999, -999, 0.657745, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[2], 16.2885, 15.5306, 14.8448, 18.345, 15.748, 16.272, 16.1103, 11.4444, 16.2411, 17.2744, 14.3509, 15.5051, 14.9657, 16.8655, 17.5437 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[3], 28.1078, 28.6507, 26.6917, 22.1132, 20.7686, -999, 25.52, 22.5956, -999, -999, 27.1077, 25.2475, 28.4725, -999, 29.5197 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[4], 27.2339, 26.6179, 25.0103, 26.9736, -999, 27.3079, 25.9764, -999, -999, -999, 28.1574, 27.9244, 28.044, 28.4966, 27.7293 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[5], 18.2775, 15.1546, 14.4757, -999, -999, 20.5039, 18.1675, -999, 14.6506, -999, -999, -999, -999, 11.739, 19.9561 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[6], -0.879364, -999, -999, -999, -999, -4.99518, 2.6944, 8.98975, 2.61905, -999, -999, -999, -999, -999, -999 /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[7], -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999, -999
As does the DDS response:
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" Dataset { Float32 /HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[8][15]; } GSSTF_NCEP.3.2008.01.01.he5;
However, the DMR request fails (and with a VERY interesting error message):
curl "http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dmr?dap4.ce=/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m%5b0:100:719%5d%5b0:100:1439%5d" <?xml version="1.0" encoding="ISO-8859-1"?>
1 libdap exception building response: error_code = 1001: libdap error transmitting DMR: /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m[0:100:719][0:100:1439]: The variable /HDFEOS.GRIDS.NCEP/Data was not found in the dataset (id). eoshelp@hdfgroup.org >On Feb 3, 2016, at 3:25 PM, H. Joe Lee notifications@github.com<mailto:notifications@github.com> wrote:
Go to http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.html
Select /HDFEOS/GRIDS/NCEP/Data Fields/Tair_2m
Press "Binary (DAP2) Object".
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5-2.dods file with text editor, you will see:
Error { code = 1005; message = "libdap error transmitting DataDDS: Constraint expression parse error: No such identifier in dataset: /HDFEOS/GRIDS/NCEP/Data"; };
PyDap will throw an error accordingly when it tries to test EnableCF=false.
— Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
— Reply to this email directly or view it on GitHubhttps://github.com/OPENDAP/hdf5_handler/issues/4#issuecomment-179536275. — Reply to this email directly or view it on GitHub.
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
= = = Nathan Potter ndp at opendap.org OPeNDAP, Inc. +1.541.231.3317
I got the file:
http://54.174.38.12/thredds/fileServer/testAll/hpd/GSSTF_NCEP.3.2008.01.01.he5
And tried it on my dev system against the current master branch of everything and it works fine.
The Data request form retrieves the correct data using this URL:
http://localhost:8080/opendap/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]
And using curl I can pipe that URL into getdap and I get the expected result.
curl -s "http://localhost:8080/opendap/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5.dods?/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]" | getdap -M -s -
The fact that this is not tested in the hdf5_handler tests is a problem and it would be great if it where. The BES command that results in dereferencing that URL might look like:
<?xml version="1.0" encoding="UTF-8"?>
<bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="http-bio-8080-exec-7:28">
<bes:setContext name="bes_timeout">40</bes:setContext>
<bes:setContext name="xdap_accept">2.0</bes:setContext>
<bes:setContext name="dap_explicit_containers">no</bes:setContext>
<bes:setContext name="errors">xml</bes:setContext>
<bes:setContext name="max_response_size">0</bes:setContext>
<bes:setContainer name="catalogContainer" space="catalog">/data/hdf5/GSSTF_NCEP.3.2008.01.01.he5</bes:setContainer>
<bes:define name="d1" space="default">
<bes:container name="catalogContainer">
<bes:constraint>/HDFEOS/GRIDS/NCEP/Data%20Fields/Tair_2m[0:100:719][0:100:1439]</bes:constraint>
</bes:container>
</bes:define>
<bes:get type="dods" definition="d1" />
</bes:request>
Modulo the value of the <setContainer>
element: /data/hdf5/GSSTF_NCEP.3.2008.01.01.he5
In general I think a good thing to do would be to verify that a suspected bug is still a problem with the current code before creating a bug report because often, as in this case, the issue has been fixed.
I upgraded the server to Hyrax 1.13 and I still can't get DAP2 binary response.
The version response doesn't look like you have completed the upgrade:
<Hyrax version="1.12.2" />
<OLFS version="1.14.1" />
What did you do? Did you get the release from the web page or did you get pieces from GitHub?
I got it from website and copied new olfs into webapps/. Let me try restarting the server and deleting old content/ directory.
You should read the installation instructions (and don't delete the content directory if you have modified it). Because the OLFS has changed significantly in the way that it locates it's configuration.
I just looked at http://54.174.38.12:8888/opendap/version and I see that it's now up-to-date in the version.
But now it appears from the structure of: http://54.174.38.12:8888/opendap/data/HPD/hpd/GSSTF_NCEP.3.2008.01.01.he5.dds That the configuration parameter:
H5.EnableCF=true
Is that what you intend?
I'm sorry for the late response. I was in a meeting and you saw the response of from the default olfs.xml server configuration. After adjusting port in olfs.xml to talk to a different BES, I could get the desired output of hdf5_handler with H5.EnableCF=false. You can check the above URL again.
By the way, I could confirm that this bug is fixed with the latest OLFS. Thanks for fixing it in a new release. I can resume my Hyrax vs. THREDDS testing with EnableCF=true/false.
Thanks for the follow up Joe, I'm glad you are getting the same results I am now.
Question: Did you set the OLFS_CONFIG_DIR
environment variable or did you create /etc/olfs
to localize the OLFS configuration?
I just modified webapps/opendap/olfs.xml directly. It was interesting to see that content/opendap is not created any more.
What you did will work for awhile, but will not persist. If the war file is unpacked again or a new one installed all of your changes will be lost. Please consider reading: http://docs.opendap.org/index.php/Hyrax_-_OLFS_Configuration#OLFS_Configuration_Location
And use either the OLFS_CONFIG_DIR
environment variable or the well known location /etc/olfs
to store your localized configuration, otherwise you may find the server behaving unpredictably down the road.
These changes where made because most user (NASA especially) install Tomcat from YUM and the content
directory strategy was broken for those systems.
This bug is found with PyDAP client. Hyrax version is http://54.174.38.12:8888/opendap/version
The download begins but if you open the GSSTF_NCEP.3.2008.01.01.he5.dods file with text editor, you will see:
PyDap will throw an error accordingly when it tries to test EnableCF=false.