Can't import CliMAF when in a read-only directory !

jypeter commented 3 years ago

I got a surprising error. After investigating, it seems that CliMAF tries to create a climaf.log file at import time in whatever the current directory is, and fails of course if the user doesn't have write access.

I guess the log file should be created in the home directory, the cache, the tmpdir, the /tmp dir, or somewhere that makes sense (if we want to retrieve the log file) and will not generate an error

Example below: I initialize CliMAF, and go to @jservonnat home dir

[jypmce@ciclad-ng ~]$ module load climaf/2.0.0-python3.6
Loading climaf/2.0.0-python3.6
  Loading requirement: nco/4.5.2 ncl/6.3.0 cdo/1.9 netcdf4/4.3.3.1-gfortran

[jypmce@ciclad-ng ~]$ cd /home/jservon

And then I start python and try to import CliMAF...

[jypmce@ciclad-ng jservon]$ python
Python 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from climaf.api import *
python => 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0]
---
Required softwares to run CliMAF => you are using the following versions/install           ations:
ncl 6.6.2 => /modfs/modtools/miniconda3/envs/analyse_3.6_test/bin/ncl
cdo 1.9.6 => /opt/nco/1.9/bin/cdo
nco (ncks) 4.5.2 => /opt/nco-4.5.2/bin/ncks
ncdump such => /prodigfs/ipslfs/dods/jservon/miniconda/envs/cesmep_env/bin/ncdum           p
Check stamping requirements
nco (ncatted) found -> /opt/nco-4.5.2/bin/ncatted
convert found -> /usr/bin/convert
pdftk found -> /usr/bin/pdftk
exiv2 found -> /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.           0.0/bin/exiv2
---
CliMAF version = 2.0.0
CliMAF install => /ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_           V2.0.0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/cli           maf/__init__.py", line 81, in <module>
    clogging.clog_file(os.getenv("CLIMAF_LOGFILE_LEVEL", "info"))
  File "/ciclad-home/jservon/Evaluation/CliMAF/climaf_installs/climaf_V2.0.0/env           /clogging.py", line 87, in clog_file
    fh = logging.FileHandler(logdir + "/" + 'climaf.log', mode='w')
  File "/modfs/modtools/miniconda3/envs/analyse_3.6_test/lib/python3.6/logging/_           _init__.py", line 1032, in __init__
    StreamHandler.__init__(self, self._open())
  File "/modfs/modtools/miniconda3/envs/analyse_3.6_test/lib/python3.6/logging/_           _init__.py", line 1061, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/home/jservon/climaf.log'
>>>

senesis commented 3 years ago

CliMAF creates the logfile in the directory indicated by environment variable CIMAF_LOG_DIR (which defaullts to "./"). See relevant doc paragraph

jypeter commented 3 years ago

Oh, OK! Well, Running CliMAF interactively below the paragraph you mention indeed says please first make sur you have write permission in the current directory (used for some log files)

But it would be nice (for the users) to check (at the beginning of a script/session) if whatever is specified for CLIMAF_LOG_DIR (default or not) is writable, and abort with an error message about this otherwise.

It would also be nice if there was a clog_file parameter that you could use to completely disable the log file

BTW:

there is no documentation for clog and clog_file
I have searched the documentation (possibly too quickly) for verbosity, severity and critical, but I have not found anything defining the different levels. Are they only defined in the example notebooks? I was wondering if there was a special level that I could use for completely disabling the log file output (e.g. something like clog_file('none'))

jypeter commented 3 years ago

I have solved my problem by adding the following code to my script before importing CliMAF

# Check the WRITE access of the directory that will be used for
# writing the CliMAF log file, and silently specify a writable
# directory if the initial directory is NOT writable
#
# CliMAF uses the directory specified in the CLIMAF_LOG_DIR
# environment variable, if defined, or the current directory
#
# If we don't do this, and a user executes a script importing CliMAF
# in a directory where (s)he has no write access, CliMAF will generate
# a PermissionError traceback that may confuse the user
#
# https://github.com/rigoudyg/climaf/issues/192
climaf_log_dir = os.environ.get('CLIMAF_LOG_DIR', os.path.curdir)
# print('Current CliMAF log directory =', climaf_log_dir)
if not os.access(climaf_log_dir, mode=os.W_OK):
    # NO write-access, we force the log directory to be '/tmp'
    os.environ['CLIMAF_LOG_DIR'] = '/tmp'
    # print('New CliMAF log directory', os.environ['CLIMAF_LOG_DIR'])

# Import CliMAF and specify some CliMAF settings
from climaf.api import *

senesis commented 3 years ago

BTW:

* there is no documentation for [clog and clog_file](https://climaf.readthedocs.io/en/latest/functions_utilities.html?highlight=LOG#clog-tune-verbosity)

There was a bug in the doc source file. This is fixed by #216

* I have searched the documentation (possibly too quickly) for `verbosity`, `severity` and `critical`, but I have not found anything defining the different levels. Are they only defined in the example notebooks? I was wondering if there was a special level that I could use for completely disabling the log file output (e.g. something like `clog_file('none')`)

The relevant level to use is 'crtitical', because it means that CliMAF can't work normally from meeting that level of issues

senesis commented 3 years ago

Pull request #216

allows to issue an explicit error message for non-writeable logifle dir
fixes doc issue for clog and clog_file

Severity levels are Python's logging standards (see https://docs.python.org/3.6/library/logging.html?highlight=logging#logging-levels)

rigoudyg / climaf

Can't import CliMAF when in a read-only directory ! #192