search

OPENDAP / hyrax-docker

GNU Lesser General Public License v3.0
22 stars 12 forks source link

Slicing not working with current tomcat configuration #33

Closed stefan-wolfsheimer closed 3 years ago

stefan-wolfsheimer commented 4 years ago

I tried to use the netcdf4 library to access the opendap service. Retrieving data works fine as long as no sliced subdata sets are requested. For example:


import netCDF4
dataset = netCDF4.Dataset('[log][show=fetch]http://myservice)

# retrieving everything works
data = dataset.variables['myvar'][:] 

# slicing yields "access failure" (in this case for a 4 dimensional dataset)
data = dataset.variables['myvar'][0, 0, 0, 0]

The the root cause of this problem was non-encoded url queries sent to the tomcat server, i.e.

https://server/opendap/filename.nc.dods?t.t[5][4][4:60][8:228]

Fix that works for us: add

relaxedQueryChars="[]|{}"

to Connector configuration in https://github.com/OPENDAP/hyrax-docker/blob/master/hyrax-1.16.2/hyrax/tomcat7-server.xml#L80

(see also https://www.unidata.ucar.edu/support/help/MailArchives/netcdf/msg14675.html)

ndp-opendap commented 4 years ago

Hi Stefan,

Some time ago Tomcat started enforcing the HTTP specification on encoding URLs and query strings.

The various netcdf libraries have made the change to correctly encode the query strings.

I am wondering:

What version of the netcdf library are you using?

Thanks,

Nathan

stefan-wolfsheimer commented 4 years ago

Hi Nathan,

I tried two version of the netcdf c library:

We prefer to support standard distribution versions rather than asking the users to compile their own libraries. Since it is easy to configure on the server side, the issue is solved for us.

However, users who just want to use the containers to explore the possibilities of Hyrax might be a bit disappointed if its not working with the standard distribution netcdf packages. With no prior experience with tomcat or hyrax it took me a while to find the cause of the actual problem.

Kind regards,

Stefan

jgallagher59701 commented 4 years ago

What version of the netcdf4 library are you using? I believe, although I might be mistaken, that Unidata have addressed this ‘non-encoded chars’ issue.

James

On Sep 1, 2020, at 4:08 AM, stefan-wolfsheimer notifications@github.com wrote:

I tried to use the netcdf4 library to access the opendap service. Retrieving data works fine as long as no sliced subdata sets are requested. For example:

import netCDF4 dataset = netCDF4.Dataset('[log][show=fetch]http://myservice)

retrieving everything works

data = dataset.variables['myvar'][:]

slicing yields "access failure" (in this case for a 4 dimensional dataset)

data = dataset.variables['myvar'][0, 0, 0, 0] The the root cause of this problem was non-encoded url queries sent to the tomcat server, i.e.

https://server/opendap/filename.nc.dods?t.t[5][4][4:60][8:228] https://server/opendap/filename.nc.dods?t.t%5B5%5D%5B4%5D%5B4:60%5D%5B8:228%5D Fix that works for us: add

relaxedQueryChars="[]|{}"

to Connector configuration in https://github.com/OPENDAP/hyrax-docker/blob/master/hyrax-1.16.2/hyrax/tomcat7-server.xml#L80 https://github.com/OPENDAP/hyrax-docker/blob/master/hyrax-1.16.2/hyrax/tomcat7-server.xml#L80 (see also https://www.unidata.ucar.edu/support/help/MailArchives/netcdf/msg14675.html https://www.unidata.ucar.edu/support/help/MailArchives/netcdf/msg14675.html)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OPENDAP/hyrax-docker/issues/33, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7Q4KTT7Z6ASIGJU7Z5RTTSDTB3VANCNFSM4QRRBAYA.

-- James Gallagher jgallagher@opendap.org