Closed larsbuntemeyer closed 2 years ago
The fsspec
package uses aoihttp
under the hood to retrieve the data. To use the ESGF credentials provided by LogonManager
with aiohttp
, you can do:
import ssl
import fsspec
import xarray as xr
from pyesgf.logon import LogonManager
# logon
manager = LogonManager()
if not manager.is_logged_on():
myproxy_host = 'esgf-data.dkrz.de'
manager.logon(hostname=myproxy_host, interactive=True, bootstrap=True)
# create SSL context
sslcontext = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
sslcontext.load_verify_locations(capath=manager.esgf_certs_dir)
sslcontext.load_cert_chain(manager.esgf_credentials)
# open file
download_url = 'http://cordexesg.dmi.dk/thredds/fileServer/cordex_general/cordex/output/EUR-11/DMI/ECMWF-ERAINT/evaluation/r1i1p1/DMI-HIRHAM5/v1/mon/tas/v20140620/tas_EUR-11_ECMWF-ERAINT_evaluation_r1i1p1_DMI-HIRHAM5_v1_mon_198901-199012.nc'
with fsspec.open(download_url, ssl=sslcontext) as file:
ds = xr.open_dataset(file)
print(ds)
<xarray.Dataset>
Dimensions: (rlat: 412, rlon: 424, time: 24, bounds: 2)
Coordinates:
lat (rlat, rlon) float64 ...
lon (rlat, rlon) float64 ...
* rlat (rlat) float64 -23.38 -23.26 -23.16 ... 21.61 21.73 21.83
* rlon (rlon) float64 -28.38 -28.26 -28.16 ... 17.93 18.05 18.16
* time (time) datetime64[ns] 1989-01-16T12:00:00 ... 1990-12-16T12...
Dimensions without coordinates: bounds
Data variables:
rotated_pole |S1 ...
tas (time, rlat, rlon) float32 ...
time_bnds (time, bounds) datetime64[ns] ...
height float64 ...
Attributes: (12/23)
CDI: Climate Data Interface version 1.5.3 (htt...
history: Thu Nov 14 20:02:22 2013: /usr/local/bin/...
institution: Danish Meteorological Institute
Conventions: CF-1.6
tracking_id: 108d904e-e9e9-464b-85a7-dba596f6b5bc
contact: obc@dmi.dk
... ...
experiment_id: evaluation
driving_experiment_name: evaluation
driving_experiment: ECMWF-ERAINT,evaluation,r1i1p1
driving_model_id: ECMWF-ERAINT
CORDEX_domain: EUR-11
NCO: 4.0.9
Thanks @bouweandela ! This is very much appreciated and solves my problem 100%!
Hi everyone, i have one more thing where I am a little lost. I can access ESGF url via
pyesgf
and it works fine for me with opendap. However, I access CORDEX datasets regularly which require a logon to ESGF for data access. It works fine with opendap and xarray if i logon and search like, e.g.,I can't access via the
download_url
, e.g.,which give a
401 Unauthorized
error...I would be greatful for any idea of how I can access CORDEX http urls. If I simply click on those http urls and login (in the webportal), I can download the files, e.g., from the browser. However, I have no experience of how to login with an open id in python for http access...