When the package ecmwflibs is available, ECCODES_DEFINITION_PATH is removed after `from gribapi import *`

[thebaptiste]

Hello

I'm using eccodes-python 1.4.1 with eccodes library 2.25.0 I have paths to local grib definitions in environment variable ECCODES_DEFINITION_PATH In my python script, after from gribapi import * or from eccodes import *, the content of the environment variable ECCODES_DEFINITION_PATH is lost, as if an unset ECCODES_DEFINITION_PATH had been done. Same thing with GRIB_DEFINITION_PATH. This was not the case with my previous usage of eccodes-python 1.3.3 with eccodes library 2.23.0

My python script :

import os
from gribapi import *
print (os.environ['ECCODES_DEFINITION_PATH'])

Best regards

[shahramn]

I just tried it and it works for me

import os
print ("Before ECCODES_DEFINITION_PATH ", os.environ['ECCODES_DEFINITION_PATH'])
print ("Before GRIB_DEFINITION_PATH ",    os.environ['GRIB_DEFINITION_PATH'])

from gribapi import *
#from eccodes import *

print ("After ECCODES_DEFINITION_PATH ", os.environ['ECCODES_DEFINITION_PATH'])
print ("After GRIB_DEFINITION_PATH ",    os.environ['GRIB_DEFINITION_PATH'])

I run with: ECCODES_DEFINITION_PATH=edp GRIB_DEFINITION_PATH=gdp python3 test_env.py

Tried with importing eccodes as well. No problem

[thebaptiste]

May be there is something wrong in my environment or I missed something in my build of eccodes library...

I'm on RockyLinux 8.5 with python 3.9.7

[thebaptiste]

Maybe something weird in eccodes include file eccodes_config.h :

/* #undef HAVE_MEMFS */

#ifdef HAVE_MEMFS
#undef ECCODES_DEFINITION_PATH
#define ECCODES_DEFINITION_PATH "/MEMFS/definitions"
#undef ECCODES_SAMPLES_PATH
#define ECCODES_SAMPLES_PATH "/MEMFS/samples"
#undef ECCODES_IFS_SAMPLES_PATH
#define ECCODES_IFS_SAMPLES_PATH "/MEMFS/ifs_samples"
#endif

and gribapi.gribapi.codes_definition_path()returns /MEMFS/definitions

[shahramn]

The MEMFS feature is described here: https://confluence.ecmwf.int/pages/viewpage.action?pageId=143037711

[thebaptiste]

I didn't build eccodes with option ENABLE_MEMFS, but it should be OFF by default ? My build options are : -DENABLE_PYTHON=OFF -DENABLE_JPG=ON -DENABLE_NETCDF=ON -DENABLE_FORTRAN=ON

codes_infois ok but, when using eccodes-python 1.4.1, after from gribapi import *, "something" seems to undefine ECCODES_DEFINITION_PATH as if HAVE_MEMFS was defined in eccodes_config.h

[thebaptiste]

May be I should build explicitely with -DENABLE_MEMFS=OFF ?

[thebaptiste]

I think I found the reason of my problem. The python package ecmwflibs is also available in my environment (release 0.4.6, may be I should update). It's a required dependency of climetlab. So eccodes-python doesn't use my eccodes build but the libeccodes library available in ecwmflibs and I suppose this library is built with mode memfs activated ? Is there a way to force eccodes-python and other ecmwf python packages to use the local built eccodes library or at least to deactivate memfs mode (we need to use local definitions) ?

[thebaptiste]

May be it would be better to use ecmwlibs libraries only if the required libraries are not found rather than the opposite ?

[thebaptiste]

Yes, it seems that I can deactivate the usage of the libeccodes library provided by ecmwflibs by setting the environment variable ECMWFLIBS_ECCODES to my locally built eccodes library. This is not very intuitive...

[thebaptiste]

From my point of vue, it's not a good idea to activate by default the MEMFS option in the eccodes library provided by ecmwflibs.

[thebaptiste]

May be this issue is rather an ecmwflibs issue...

[floriankrb]

Yes, it seems that I can deactivate the usage of the libeccodes library provided by ecmwflibs by setting the environment variable ECMWFLIBS_ECCODES to my locally built eccodes library. This is not very intuitive...

I think this is the main issue (the only issue?): "This is not very intuitive".

Perhaps eccodes-python should warn when there are env variables and they are ignored because it chooses ecmwflibs instead? And give a hint to use ECMWFLIBS_ECCODES?

[floriankrb]

The commit https://github.com/ecmwf/ecmwflibs/commit/f97bc933e4f952670d1db60ffbe6fb92b5f93234 in ecmwflibs solves the problem that "this is not intuitive". Adding warning and help when env variables are ignored.

This issue can be closed.