search

xCDAT / xcdat

An extension of xarray for climate data analysis on structured grids.
https://xcdat.readthedocs.io/en/latest/
Apache License 2.0
101 stars 11 forks source link

[Bug]: Fix Read the Docs build with ESMF env variable not being set in v0.6.1 #574

Closed tomvothecoder closed 3 months ago

tomvothecoder commented 7 months ago

What happened?

In xCDAT <=0.6.0, xesmf was an optional dependency in the codebase and package. As a result, RTD builds were successful despite not installing xesmf.

xCDAT now lists xesmf as a required dependency. This causes issues with the RTD build because the ESMF 'ESMFMKFILE' env variable is not being set. RTD does not activate the conda environment before running other steps in the build process, resulting in failures with RTD builds for v0.6.1+.

What did you expect to happen? Are there are possible answers you came across?

I believe as long as the conda environment that has xcdat=0.6.1 and xesmf is activated, ESMFMKFILE should be set. However, we can't modify the build steps for RTD since that is all handled internally. More info in this comment.

For future reference, both of these workarounds resolve this issue:

  1. Update .readthedocs.yaml to set conda env variables: https://github.com/readthedocs/readthedocs.org/issues/5339#issuecomment-1576603529
  2. Update conf.py to set environment variables: https://github.com/conda-forge/esmf-feedstock/issues/91#issuecomment-1387279692

Minimal Complete Verifiable Example (MVCE)

No response

Relevant log output

Running Sphinx v5.3.0

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 26, in <module>
    esmfmk = os.environ["ESMFMKFILE"]
             ~~~~~~~~~~^^^^^^^^^^^^^^
  File "<frozen os>", line 679, in __getitem__
KeyError: 'ESMFMKFILE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 22, in <module>
    import esmpy as ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/__init__.py", line 112, in <module>
    from esmpy.api.esmpymanager import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/api/esmpymanager.py", line 9, in <module>
    from esmpy.interface.cbindings import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/cbindings.py", line 13, in <module>
    from esmpy.interface.loadESMF import _ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 28, in <module>
    raise ImportError('The ESMFMKFILE environment variable is not available.')
ImportError: The ESMFMKFILE environment variable is not available.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/config.py", line 350, in eval_config_file
    exec(code, namespace)
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/docs/conf.py", line 27, in <module>
    import xcdat  # noqa: I001, E402
    ^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/__init__.py", line 10, in <module>
    from xcdat.regridder.accessor import RegridderAccessor  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/__init__.py", line 1, in <module>
    from xcdat.regridder.accessor import RegridderAccessor
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/accessor.py", line 9, in <module>
    from xcdat.regridder import regrid2, xesmf, xgcm
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/xesmf.py", line 4, in <module>
    import xesmf as xe
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/__init__.py", line 4, in <module>
    from .frontend import Regridder, SpatialAverager
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/frontend.py", line 14, in <module>
    from .backend import Grid, LocStream, Mesh, add_corner, esmf_regrid_build, esmf_regrid_finalize
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 24, in <module>
    import ESMF
ModuleNotFoundError: No module named 'ESMF'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/application.py", line 202, in __init__
    self.config = Config.read(self.confdir, confoverrides or {}, self.tags)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/config.py", line 172, in read
    namespace = eval_config_file(filename, tags)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/config.py", line 363, in eval_config_file
    raise ConfigError(msg % traceback.format_exc()) from exc
sphinx.errors.ConfigError: There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 26, in <module>
    esmfmk = os.environ["ESMFMKFILE"]
             ~~~~~~~~~~^^^^^^^^^^^^^^
  File "<frozen os>", line 679, in __getitem__
KeyError: 'ESMFMKFILE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 22, in <module>
    import esmpy as ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/__init__.py", line 112, in <module>
    from esmpy.api.esmpymanager import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/api/esmpymanager.py", line 9, in <module>
    from esmpy.interface.cbindings import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/cbindings.py", line 13, in <module>
    from esmpy.interface.loadESMF import _ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 28, in <module>
    raise ImportError('The ESMFMKFILE environment variable is not available.')
ImportError: The ESMFMKFILE environment variable is not available.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/config.py", line 350, in eval_config_file
    exec(code, namespace)
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/docs/conf.py", line 27, in <module>
    import xcdat  # noqa: I001, E402
    ^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/__init__.py", line 10, in <module>
    from xcdat.regridder.accessor import RegridderAccessor  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/__init__.py", line 1, in <module>
    from xcdat.regridder.accessor import RegridderAccessor
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/accessor.py", line 9, in <module>
    from xcdat.regridder import regrid2, xesmf, xgcm
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/xesmf.py", line 4, in <module>
    import xesmf as xe
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/__init__.py", line 4, in <module>
    from .frontend import Regridder, SpatialAverager
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/frontend.py", line 14, in <module>
    from .backend import Grid, LocStream, Mesh, add_corner, esmf_regrid_build, esmf_regrid_finalize
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 24, in <module>
    import ESMF
ModuleNotFoundError: No module named 'ESMF'

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 26, in <module>
    esmfmk = os.environ["ESMFMKFILE"]
             ~~~~~~~~~~^^^^^^^^^^^^^^
  File "<frozen os>", line 679, in __getitem__
KeyError: 'ESMFMKFILE'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 22, in <module>
    import esmpy as ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/__init__.py", line 112, in <module>
    from esmpy.api.esmpymanager import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/api/esmpymanager.py", line 9, in <module>
    from esmpy.interface.cbindings import *
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/cbindings.py", line 13, in <module>
    from esmpy.interface.loadESMF import _ESMF
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/esmpy/interface/loadESMF.py", line 28, in <module>
    raise ImportError('The ESMFMKFILE environment variable is not available.')
ImportError: The ESMFMKFILE environment variable is not available.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/sphinx/config.py", line 350, in eval_config_file
    exec(code, namespace)
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/docs/conf.py", line 27, in <module>
    import xcdat  # noqa: I001, E402
    ^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/__init__.py", line 10, in <module>
    from xcdat.regridder.accessor import RegridderAccessor  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/__init__.py", line 1, in <module>
    from xcdat.regridder.accessor import RegridderAccessor
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/accessor.py", line 9, in <module>
    from xcdat.regridder import regrid2, xesmf, xgcm
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/checkouts/v0.6.1/xcdat/regridder/xesmf.py", line 4, in <module>
    import xesmf as xe
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/__init__.py", line 4, in <module>
    from .frontend import Regridder, SpatialAverager
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/frontend.py", line 14, in <module>
    from .backend import Grid, LocStream, Mesh, add_corner, esmf_regrid_build, esmf_regrid_finalize
  File "/home/docs/checkouts/readthedocs.org/user_builds/xcdat/conda/v0.6.1/lib/python3.11/site-packages/xesmf/backend.py", line 24, in <module>
    import ESMF
ModuleNotFoundError: No module named 'ESMF'

Anything else we need to know?

Environment

xcdat=0.6.1

lee1043 commented 7 months ago

@tomvothecoder The same error just occurred to me with the v0.6.1. What would be the solution? Should we consider xesmf as an optional dependency?

tomvothecoder commented 7 months ago

@tomvothecoder The same error just occurred to me with the v0.6.1. What would be the solution? Should we consider xesmf as an optional dependency?

Are you running xcdat=0.6.1 on Linux/Mac? I am not getting this issue with a fresh install of xCDAT in a new environment on Mac, Linux, and my personal Windows machine. I think as long as the conda environment is activated, this file should be found.

mamba create -n xcdat_061 xcdat=0.6.1
mamba activate xcdat_061
>>> import xesmf
>>> import xcdat
>>> import os
>>> print(os.environ["ESMFMKFILE"])
'/opt/miniconda3/envs/xcdat_061/lib/esmf.mk'

Separate issue with ESMF/esmpy v8.5 and v8.6

Additionally, I just found out that ESMF/esmpy v8.5 and v8.6 are not released on conda-forge yet because of issues with the Windows build. They are also considering dropping the Windows support due to GCC build issues (comment). These version include a fix to automatically guess the ESMFMK path (https://github.com/esmf-org/esmf/pull/151) if the env variable is not set, which happens if the conda env is not activated.

I think the safe bet might be to make xESMF an optional dependency again in v0.6.2+. We can also just wait to see what ESMF/esmpy devs decide to do. FYI @xCDAT/core-developers, sorry for the quick release of v0.6.1 without diving deeper to these non-obvious issues. ESMF/esmpy/xesmf has been challenging to work with on the build side of things.

tomvothecoder commented 7 months ago

As long as the conda environment with esmf/esmpy/xesmf is activated, the ESMFMKFILE env var will be set. This issue has been around and we just haven’t come across it. Making xesmf a required dependency doesn’t really change anything.

The new behaviour in 8.4 was to depend on the env variable. The ESMpy conda package was updated so that this env var is set upon environment activation. However, there are many cases in which a conda env is used without being activated:

when used as a kernel in a jupyter lab/notebook instance
when used to build documentation on ReadTheDocs
when used as a kernel in some fancy IDEs (PyCharm for example)

-- Source: https://github.com/ESMValGroup/ESMValCore/issues/2001

I have tried in many ways to set that var in the RTD environment - to no avail, the problem is that RTD creates an env per doc build (the env is named either latest, or stable, or the number of the PR that triggered it) and doesn't do conda activate [env] explicitly so that variable is not set; trying to set it via the .readthedocs.yaml or conf.py conf files is like trying to talk to Captain Ahab that is currently stuck inside Moby Dick ie no control over the RTD's conda gubbins which are all under the hood. If anyone's managed to do this (specifically for the RTD case) I'd be very grateful for a workaround/fix 🍺 -- Source: https://github.com/ESMValGroup/ESMValCore/issues/2001](https://github.com/conda-forge/esmf-feedstock/issues/91#issuecomment-1505270915)

lee1043 commented 7 months ago

Are you running xcdat=0.6.1 on Linux/Mac? I am not getting this issue with a fresh install of xCDAT in a new environment on Mac, Linux, and my personal Windows machine. I think as long as the conda environment is activated, this file should be found.

@tomvothecoder I made a fresh env and the issue is not occurring. It might caused by some conflict from my old env, which I couldn't identify the specific conflicted one.

lee1043 commented 5 months ago

I am reopening this issue as I am getting this error from Nimbus jupyterhub when importing pcmdi_metrics, which is coming from xesmf underneath of xcdat.

@jasonb5 do you have any idea how to address this?

tomvothecoder commented 5 months ago

Workaround (this must be broken into two Jupyter cells if using Jupyter):

Cell one:

import os
os.environ['ESMFMKFILE'] = 'conda-envs/xcdat/lib/esmf.mk'

Cell two:

import xcdat as xc
tomvothecoder commented 4 months ago

Was this ever resolved? @lee1043

lee1043 commented 4 months ago

I opened this for the Nimbus jupyter hub (NOT binder hub), but as it may not xCDAT issue, I am okay to close this issue. I expect the Nimbus jupyter hub issue should be solved when the env updated. @jasonb5 could you confirm my theory?

pochedls commented 4 months ago

@lee1043 – does this work (three comments up)?

If so, maybe that should be added to the docs somewhere.

tomvothecoder commented 3 months ago

This is fixed