search

Unidata / netcdf4-python

netcdf4-python: python/numpy interface to the netCDF C library
http://unidata.github.io/netcdf4-python
MIT License
757 stars 264 forks source link

Error while executing python script to retrieve netCDF4 from OpenDap. Error Message: Error:curl error: Problem with the SSL CA cert (path? access rights?) running inside a docker file #1199

Open advapiIT opened 2 years ago

advapiIT commented 2 years ago

Hello, I have to say I'm new to NetCDF4 and python in general, I've been asked to create a tool that converts from opendap file to .nc in order to process them. I've got my application (.NET 6) that runs and execute a python script that performs the conversion. On windows env everything works fine.

When I run inside a container I got the following error

Error while executing python script to retrieve netCDF4 from OpenDap. Error Message: Error:curl error: Problem with the SSL CA cert (path? access rights?)

The URL I'm trying to get is https://thredds.met.no/thredds/dodsC/arcticdata/obsSynop/04320 .

Is there a way I can tell NetCDF to accept all certificates? If I do it from inside the container

curl https://thredds.met.no/thredds/dodsC/arcticdata/obsSynop/04320.html it shows the data...so curl seems to work and doen't show anything about certficiate issue

I'm using latest version of NetCDF 1.6.1 and python 3.9

I do this inside my container to install python3/netdf

RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN pip3 install netCDF4
jswhit commented 2 years ago

works for me

>>> from netCDF4 import Dataset
>>> nc = Dataset('https://thredds.met.no/thredds/dodsC/arcticdata/obsSynop/04320')
>>> nc
<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, file format DAP2):
    featureType: timeSeries
    title: SYNOP data from station DANMARKSHAVN
    abstract: Synoptic meteorological measurements from DANMARKSHAVN
    institution: Norwegian Meteorological Institute
    contact: o.godoy@met.no
    PI_name: Øystein Godøy
    Conventions: CF-1.4
    activity_type: Land station
    topiccategory: ClimatologyMeteorologyAtmosphere
    keywords: Atmospheric Observation Temperature Humidity Clouds Pressure Precipitation Wind SYNOP
    gcmd_keywords: Atmosphere > Atmospheric Pressure > Surface Pressure
Atmosphere > Atmospheric Temperature > Surface Air Temperature
Atmosphere > Atmospheric Water Vapor > Humidity
Atmosphere > Clouds > Cloud Types
Atmosphere > Clouds > Cloud Amount/Frequency
Atmosphere > Precipitation > Precipitation Amount
Atmosphere > Atmospheric Winds > Surface Winds
Cryosphere > Snow/Ice > Snow Depth
    project_name: ACCESS
    area: Northern Hemisphere
    product_name: SYNOP
    distribution_statement: Free
    history: 2022-01-02 revision
    southernmost_latitude: 76.76944
    northernmost_latitude: 76.76944
    westernmost_longitude: -18.66806
    easternmost_longitude: -18.66806
    start_date: 2021-01-01 00:00:00 UTC
    stop_date: 2021-12-31 23:00:00 UTC
    DODS.strlen: 5
    DODS.dimName: name_strlen
    dimensions(sizes): maxStrlen64(64), obs(16955)
    variables(dimensions): float32 latitude(), float32 longitude(), float32 alt(), |S1 station_wmonr(maxStrlen64), float64 obs(obs), float32 TA(obs), float32 TD(obs), float32 TAX_12(obs), float32 TAN_12(obs), float32 TW(obs), float32 PR(obs), float32 PO(obs), float32 PP(obs), int32 AA(obs), float32 RR_1(obs), float32 RR_6(obs), float32 RR_12(obs), float32 RR_24(obs), float32 SA(obs), int32 EE(obs), float32 FF(obs), float32 DD(obs), float32 FG_010(obs), float32 FG(obs), float32 VV(obs), float32 NN(obs), float32 HL(obs), int32 CL(obs), int32 CM(obs), int32 CH(obs), float32 UU(obs), int32 WW(obs), int32 W1(obs), int32 W2(obs), float32 HW(obs), float32 PW(obs), float32 HWA(obs), float32 PWA(obs)
    groups: 
>>>

Maybe the netcdf lib in your container was not built with curl support, or the linked curl lib doesn't have ssl support?

advapiIT commented 2 years ago

but why reverting to 1.5.8 for netcdf lib it works?

On Wed, Oct 12, 2022 at 5:25 PM Jeff Whitaker @.***> wrote:

works for me

from netCDF4 import Dataset nc = Dataset('https://thredds.met.no/thredds/dodsC/arcticdata/obsSynop/04320') nc <class 'netCDF4._netCDF4.Dataset'> root group (NETCDF3_CLASSIC data model, file format DAP2):

featureType: timeSeries

title: SYNOP data from station DANMARKSHAVN

abstract: Synoptic meteorological measurements from DANMARKSHAVN

institution: Norwegian Meteorological Institute

contact: ***@***.***

PI_name: Øystein Godøy

Conventions: CF-1.4

activity_type: Land station

topiccategory: ClimatologyMeteorologyAtmosphere

keywords: Atmospheric Observation Temperature Humidity Clouds Pressure Precipitation Wind SYNOP

gcmd_keywords: Atmosphere > Atmospheric Pressure > Surface Pressure

Atmosphere > Atmospheric Temperature > Surface Air Temperature Atmosphere > Atmospheric Water Vapor > Humidity Atmosphere > Clouds > Cloud Types Atmosphere > Clouds > Cloud Amount/Frequency Atmosphere > Precipitation > Precipitation Amount Atmosphere > Atmospheric Winds > Surface Winds Cryosphere > Snow/Ice > Snow Depth

project_name: ACCESS

area: Northern Hemisphere

product_name: SYNOP

distribution_statement: Free

history: 2022-01-02 revision

southernmost_latitude: 76.76944

northernmost_latitude: 76.76944

westernmost_longitude: -18.66806

easternmost_longitude: -18.66806

start_date: 2021-01-01 00:00:00 UTC

stop_date: 2021-12-31 23:00:00 UTC

DODS.strlen: 5

DODS.dimName: name_strlen

dimensions(sizes): maxStrlen64(64), obs(16955)

variables(dimensions): float32 latitude(), float32 longitude(), float32 alt(), |S1 station_wmonr(maxStrlen64), float64 obs(obs), float32 TA(obs), float32 TD(obs), float32 TAX_12(obs), float32 TAN_12(obs), float32 TW(obs), float32 PR(obs), float32 PO(obs), float32 PP(obs), int32 AA(obs), float32 RR_1(obs), float32 RR_6(obs), float32 RR_12(obs), float32 RR_24(obs), float32 SA(obs), int32 EE(obs), float32 FF(obs), float32 DD(obs), float32 FG_010(obs), float32 FG(obs), float32 VV(obs), float32 NN(obs), float32 HL(obs), int32 CL(obs), int32 CM(obs), int32 CH(obs), float32 UU(obs), int32 WW(obs), int32 W1(obs), int32 W2(obs), float32 HW(obs), float32 PW(obs), float32 HWA(obs), float32 PWA(obs)

groups:

Maybe the netcdf lib in your container was not built with curl support, or the linked curl lib doesn't have ssl support?

— Reply to this email directly, view it on GitHub https://github.com/Unidata/netcdf4-python/issues/1199#issuecomment-1276361256, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDZBJU3DL44QBKSKIKTXSTWC3J5ZANCNFSM6AAAAAARDCRFQU . You are receiving this because you authored the thread.Message ID: @.***>

jswhit commented 2 years ago

Must be an issue with the linked netcdf-c and/or curl libs - has nothing to do with the python interface itself. Can you try with the conda package?

dopplershift commented 2 years ago

@advapilT are you aware that nccopy (comes with the netCDF C library) can do this for you already?

nccopy -V TA,TD https://thredds.met.no/thredds/dodsC/arcticdata/obsSynop/04320 mydata.nc