search

NCAR / wrf-python

A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model.
https://wrf-python.readthedocs.io
Apache License 2.0
410 stars 155 forks source link

wrf-python installed from conda-forge produces import error #110

Open wxguy opened 4 years ago

wxguy commented 4 years ago

Installed wrf-pyhton through conda-forge on windows 7 64 bit. Post installation, checked if the package is installed correctly using following command:-

 python -c "import wrf"

and it worked without an error. However, when I ran the script the following error is produced:-

Python 3.7.6 (default, Jan  8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import wrf
Traceback (most recent call last):
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\__init__.py", line 6, in <module>
    from . import api
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\api.py", line 1, in <module>
    from .config import (xarray_enabled, disable_xarray, enable_xarray,
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\config.py", line 6, in <module>
    from ._wrffortran import (fomp_enabled, fomp_set_num_threads,
ImportError: DLL load failed: The specified path is invalid.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\__init__.py", line 18, in <module>
    from . import api
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\api.py", line 1, in <module>
    from .config import (xarray_enabled, disable_xarray, enable_xarray,
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\config.py", line 6, in <module>
    from ._wrffortran import (fomp_enabled, fomp_set_num_threads,
ImportError: DLL load failed: The specified path is invalid.

I am not sure what causes the problem. Could someone help on this t rectify...

Edit:-

Just to clarify that the same works on Linux version of miniconda.

Edit-2

Uninstalled latest version of wrf-python and installed older version (1.3.0). This time I am getting following error:-

Traceback (most recent call last):
  File "insdvp_app.py", line 30, in <module>
    import utils
  File "Y:\project\insdvpgit\insdvp\utils.py", line 34, in <module>
    from wrf import interplevel
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\__init__.py", line 16, in <module>
    from . import api
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\api.py", line 1, in <module>
    from .config import (xarray_enabled, disable_xarray, enable_xarray,
  File "C:\Users\Sundar\miniconda3\envs\insdvp\lib\site-packages\wrf\config.py", line 6, in <module>
    from ._wrffortran import (fomp_enabled, fomp_set_num_threads,
ImportError: DLL load failed: The specified path is invalid.

Edit-3

Fiddled around with package from https://www.lfd.uci.edu/~gohlke/pythonlibs/. This time it successfully installed (version 1.2.0 as it was the latest at that point of time) and imported without an error. However, while reading the wrfout file, the script closed without producing an error.

To check if the problem exist in window 10, I installed the module on a freshly installed miniconda. This time it did not show any error as I have observed in windows 7. However, It simply closes without producing any error when reading a wrfout netcdf file as seen in latest try on windows 7.

Edit-4 Checked the wheel package today again at https://www.lfd.uci.edu/~gohlke/pythonlibs/#wrf_python and found the latest version 1.3.2. Uninstalled the conda version and installed whell package from link. Finally, It worked :-)

Something is wrong with conda install on windows. You guys may want to chack it out...

honnorat commented 4 years ago

I don't know if it's related, but Windows builds in this PR https://github.com/conda-forge/wrf-python-feedstock/pull/45 prevent wrf-python being available for conda py3.8 (cf. https://conda-forge.org/status/)

wxguy commented 4 years ago

I don't know if it's related, but Windows builds in this PR conda-forge/wrf-python-feedstock#45 prevent wrf-python being available for conda py3.8 (cf. https://conda-forge.org/status/)

I continued with my urge to install wrf-python on windows 7/10 on Pthon 3.8.2. I am happy to report here that I am able to install wrf-python successfully from the ground up i.e. installing latest Python 3.8.2 up to running my application and packaging through PyInstaller. For the benefit of others, here is the check of list:-

  1. Install Python. I used 3.8.2 (see point no 2 before downloading)

  2. Download wheel packages for numpy-mkl, scipy, cartopy and wrf-python from https://www.lfd.uci.edu/~gohlke/pythonlibs/. I strongly recommend that you choose your Python version based on the package of your interest available in this repo. In my case wrf-python for Python 3.8 was not available until recently. Used pip to install each packages in sequence of numpy-mkl --> scipy --> cartopy --> wrf-python.

  3. Other standard packages I use such as xarray, pandas, netCDF4, h5py etc can be directly installed from pip command from pip server.

  4. I did some development and used pyinstaller (development version) to pack the application in one dir mode. The application did not start initially and complained about missing mkl dll. Copied those mkl_*.dll libraries from numpy install directory under DLLs to root directory of application, generated by pyinstaller. All worked thereafter.

The main reason I did not want to install conda was to reduce the distribution size of my application. But it turned out that without numpy-mkl package, wrf-python will not install which is quite huge in size. Despite all this, the overall pacakage size has come down drastically as compared with miniconda. More and more complicated packages are now a days providing Python wheel for their packages. May be this is the time to upload wrf-python as wheel to pip to reach more audience. Thank you.

wxguy commented 4 years ago

I don't know if it's related, but Windows builds in this PR conda-forge/wrf-python-feedstock#45 prevent wrf-python being available for conda py3.8 (cf. https://conda-forge.org/status/)

I continued with my urge to install wrf-python on windows 7/10 on Pthon 3.8.2. I am happy to report here that I am able to install wrf-python successfully from the ground up i.e. installing latest Python 3.8.2 up to running my application and packaging through PyInstaller. For the benefit of others, here is the check of list:-

1. Install Python. I used 3.8.2 (see point no 2 before downloading)

2. Download wheel packages for `numpy-mkl`, `scipy`, `cartopy` and `wrf-python` from https://www.lfd.uci.edu/~gohlke/pythonlibs/. I strongly recommend that you choose your Python version based on the package of your interest available in this repo. In my case wrf-python for Python 3.8 was not available until recently. Used pip to install each packages in sequence of `numpy-mkl` --> `scipy` --> `cartopy` --> `wrf-python`.

3. Other standard packages I use such as `xarray`, `pandas`, `netCDF4`, `h5py` etc can be directly installed from `pip` command from `pip` server.

4. I did some development and used pyinstaller (development version) to pack the application in one dir mode. The application did not start initially and complained about missing mkl dll. Copied those mkl_*.dll libraries from numpy install directory under `DLLs` to root directory of application, generated by pyinstaller. All worked thereafter.

The main reason I did not want to install conda was to reduce the distribution size of my application. But it turned out that without numpy-mkl package, wrf-python will not install which is quite huge in size. Despite all this, the overall pacakage size has come down drastically as compared with miniconda. More and more complicated packages are now a days providing Python wheel for their packages. May be this is the time to upload wrf-python as wheel to pip to reach more audience. Thank you.

At least, word of caution for those installing from pip may be mentioned in the documentation that they need to install numpy-mkl package only to install wrf-python.

Doubleegg-commander commented 3 years ago

Hello, when I use the wrf-python 1.3.2 version provided by https://www.lfd.uci.edu/~gohlke/pythonlibs/ under the environment of python3.7.4 under win10, the error you mentioned still appears . I would like to ask if you have any good solutions. Thank you

samuelchodur commented 2 years ago

Hello, when I use the wrf-python 1.3.2 version provided by https://www.lfd.uci.edu/~gohlke/pythonlibs/ under the environment of python3.7.4 under win10, the error you mentioned still appears . I would like to ask if you have any good solutions. Thank you

I believe the issue may be caused by installing the OpenBLAS version of NumPy as opposed to Intel MKL version. When I installed using the MKL wheel from https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy I was able to successfully import wrf-python.