search

Unidata / netcdf-c

Official GitHub repository for netCDF-C libraries and utilities.
BSD 3-Clause "New" or "Revised" License
512 stars 262 forks source link

cdump on protected esgf files #137

Open doutriaux1 opened 8 years ago

doutriaux1 commented 8 years ago

we are trying to update or .dodsrc to reflect the new format, but we are unable to get it working.

first the .dodsrc

USE_CACHE=0
HTTP.VERBOSE=0
HTTP.COOKIEJAR=/export/ames4/.esg/.dods_cookies
HTTP.SSL.VALIDATE=0
HTTP.SSL.CERTIFICATE=/export/ames4/.esg/credentials.pem
HTTP.SSL.KEY=/export/ames4/.esg/credentials.pem
HTTP.SSL.CAPATH=/export/ames4/.esg/certificates

now the ncdump command

ncdump -h  http://aims3.llnl.gov/thredds/dodsC/cmip5_css02_data/cmip5/output2/INM/inmcm4/1pctCO2/mon/land/Lmon/r1i1p1/cProduct/1/cProduct_Lmon_inmcm4_1pctCO2_r1i1p1_209001-222912.nc

error is:

CURL Error: SSL connect error
curl error details: 
ncdump: http://aims3.llnl.gov/thredds/dodsC/cmip5_css02_data/cmip5/output2/INM/inmcm4/1pctCO2/mon/land/Lmon/r1i1p1/cProduct/1/cProduct_Lmon_inmcm4_1pctCO2_r1i1p1_209001-222912.nc: NetCDF: I/O failure

@sashakames says that if you register on pcmdi9 and ask for CMIP5 group you should be able to reproduce this.

@dnadeau4 feel free to take a look if you want

@DennisHeimbigner I think this one is probably for you, any hint appreciated

sashakames commented 8 years ago

You also need to fetch your client side cert from esgf. The easiest way to do this is to generate and run a wget script (but stop it once you start downloading). Myproxy password is your esgf openid password.

You need to agree to the CMIP5 licence agreement. There is a link on pcmdi9.llnl.gov to do this.

DennisHeimbigner commented 8 years ago

At some point, the curl library appears to have changed how it handles some kinds of authorization. I have been slowly working to fix these as people report them. I will move this up the stack since you are in a position to help me fix it.

WardF commented 8 years ago

@dmh is this something we should target for the 4.4.0 release or should I tag it 'future' ?

DennisHeimbigner commented 8 years ago

Let me see if we can fix it fast.

DennisHeimbigner commented 8 years ago

My immediate guess is that you are not using .netrc, which apparently is now required by libcurl. Have you seen the attached document? auth.txt

dnadeau4 commented 8 years ago

Let me try this!

dnadeau4 commented 8 years ago

So it turns out that we were using a very old version of libcurl.

RedHat 6.7 has libcurl-7.19.7-46.el6.i686 which does not have the same SSL connection especially the TLS connection. Ubuntu is using libcurl-7.38

From Redhat 6.7:

When running using Ubuntu, we got the right answer from ncdump.

Denis

doutriaux1 commented 8 years ago

@sashakames that might be something you need to bring up with esgf. @DennisHeimbigner I'll investigate some more, but is your wild guess is that there is no way to make this work again with older curl?