search

pySTEPS / pysteps

Python framework for short-term ensemble prediction systems.
https://pysteps.github.io/
BSD 3-Clause "New" or "Revised" License
466 stars 168 forks source link

Fix failing CI #250

Closed dnerini closed 2 years ago

dnerini commented 2 years ago

Fix #249.

codecov[bot] commented 2 years ago

Codecov Report

Merging #250 (ed130cb) into master (bd94785) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #250   +/-   ##
=======================================
  Coverage   80.78%   80.78%           
=======================================
  Files         140      140           
  Lines       10625    10625           
=======================================
  Hits         8583     8583           
  Misses       2042     2042
Flag Coverage Δ
unit_tests 80.78% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bd94785...ed130cb. Read the comment docs.

dnerini commented 2 years ago

doh! now windows py38 fails with a new error:

Traceback (most recent call last):
  File "fetch_pysteps_data.py", line 14, in <module>
    from pysteps.datasets import create_default_pystepsrc, download_pysteps_data
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\__init__.py", line 11, in <module>
    from . import cascade
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\cascade\__init__.py", line 5, in <module>
    from .interface import get_method
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\cascade\interface.py", line 14, in <module>
    from pysteps.cascade import decomposition, bandpass_filters
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\cascade\decomposition.py", line 76, in <module>
    from pysteps import utils
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\utils\__init__.py", line 4, in <module>
    from .cleansing import *
  File "C:\Miniconda\envs\test-environment\lib\site-packages\pysteps\utils\cleansing.py", line 17, in <module>
    import scipy.spatial
  File "C:\Miniconda\envs\test-environment\lib\site-packages\scipy\__init__.py", line 136, in <module>
    from . import _distributor_init
  File "C:\Miniconda\envs\test-environment\lib\site-packages\scipy\_distributor_init.py", line 59, in <module>
    WinDLL(os.path.abspath(filename))
  File "C:\Miniconda\envs\test-environment\lib\ctypes\__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Miniconda\envs\test-environment\lib\site-packages\scipy\.libs\libbanded5x.CBCBXIAYFUWOYCSBXUA2BT6W76FYX4C7.gfortran-win_amd64.dll' (or one of its dependencies). Try using the full path with constructor syntax.

edit: fixed in ed130cb ref: https://github.com/scipy/scipy/issues/14002

dnerini commented 2 years ago

The changes introduced here are not necessary anymore since numpy 1.22 was released on conda-forge, too. Since the changes were more of a workaround than actual fix, let's not merge this but keep it here as reference in case problem appears again.

ErickMesquita commented 2 years ago

The changes introduced here are not necessary anymore since numpy 1.22 was released on conda-forge, too. Since the changes were more of a workaround than actual fix, let's not merge this but keep it here as reference in case problem appears again.

Numpy 1.22 didn't make your changes unnecessary, @dnerini . I tried with numpy 1.22.0 py38hcf66579_1 from conda-forge, but it is still throwing FileNotFoundError even though the file is right there.

I only got it to work after setting CONDA_DLL_SEARCH_MODIFICATION_ENABLE to 1

conda list short Name Version Build Channel numpy 1.22.0 py38hcf66579_1 conda-forge pandas 1.3.5 py38h5d928e2_0 conda-forge python 3.8.12 h7840368_2_cpython conda-forge scipy 1.7.3 py38ha1292f7_0 conda-forge
conda list Name Version Build Channel argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py38h294d835_1 conda-forge asttokens 2.0.5 pyhd8ed1ab_0 conda-forge attrs 21.4.0 pyhd8ed1ab_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge black 21.12b0 pyhd8ed1ab_0 conda-forge bleach 4.1.0 pyhd8ed1ab_0 conda-forge boost-cpp 1.74.0 h5b4e17d_5 conda-forge branca 0.4.2 pyhd8ed1ab_0 conda-forge brotli 1.0.9 h8ffe710_6 conda-forge brotli-bin 1.0.9 h8ffe710_6 conda-forge brotlipy 0.7.0 py38h294d835_1003 conda-forge bzip2 1.0.8 h8ffe710_4 conda-forge ca-certificates 2021.10.8 h5b45459_0 conda-forge cairo 1.16.0 hb19e0ff_1008 conda-forge certifi 2021.10.8 py38haa244fe_1 conda-forge cffi 1.15.0 py38hd8c33c5_0 conda-forge cfitsio 3.470 h0af3d06_7 conda-forge charset-normalizer 2.0.10 pyhd8ed1ab_0 conda-forge click 8.0.3 py38haa244fe_1 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge colorama 0.4.4 pyh9f0ad1d_0 conda-forge cryptography 36.0.1 py38hb7941b4_0 conda-forge curl 7.81.0 h789b8ee_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge dataclasses 0.8 pyhc8e2a94_3 conda-forge debugpy 1.5.1 py38h885f38d_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge executing 0.8.2 pyhd8ed1ab_0 conda-forge expat 2.4.3 h39d44d4_0 conda-forge fiona 1.8.20 py38hbf36ab5_2 conda-forge flit-core 3.6.0 pyhd8ed1ab_0 conda-forge folium 0.12.1.post1 pyhd8ed1ab_1 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.13.1 h1989441_1005 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.28.5 py38h294d835_0 conda-forge freetype 2.10.4 h546665d_1 conda-forge freexl 1.0.6 ha8e266a_0 conda-forge gdal 3.3.3 py38h117dad8_0 conda-forge geopandas 0.10.2 pyhd8ed1ab_1 conda-forge geopandas-base 0.10.2 pyha770c72_1 conda-forge geos 3.9.1 h39d44d4_2 conda-forge geotiff 1.7.0 h350af67_3 conda-forge gettext 0.19.8.1 ha2e2712_1008 conda-forge hdf4 4.2.15 h0e5069d_3 conda-forge hdf5 1.12.1 nompi_h2a0e4a3_103 conda-forge icu 68.2 h0e60522_0 conda-forge idna 3.3 pyhd8ed1ab_0 conda-forge importlib-metadata 4.10.1 py38haa244fe_0 conda-forge importlib_resources 5.4.0 pyhd8ed1ab_0 conda-forge intel-openmp 2022.0.0 h57928b3_3663 conda-forge ipykernel 6.7.0 py38h595d716_0 conda-forge ipython 8.0.1 py38haa244fe_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge jbig 2.1 h8d14728_2003 conda-forge jedi 0.18.1 py38haa244fe_0 conda-forge jinja2 3.0.3 pyhd8ed1ab_0 conda-forge joblib 1.1.0 pyhd8ed1ab_0 conda-forge jpeg 9d h8ffe710_0 conda-forge jsonschema 4.4.0 pyhd8ed1ab_0 conda-forge jupyter 1.0.0 py38haa244fe_7 conda-forge jupyter_client 7.1.1 pyhd8ed1ab_0 conda-forge jupyter_console 6.4.0 pyhd8ed1ab_0 conda-forge jupyter_core 4.9.1 py38haa244fe_1 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_widgets 1.0.2 pyhd8ed1ab_0 conda-forge kealib 1.4.14 h8995ca9_3 conda-forge kiwisolver 1.3.2 py38hbd9d945_1 conda-forge krb5 1.19.2 h20d022d_3 conda-forge lcms2 2.12 h2a16943_0 conda-forge lerc 3.0 h0e60522_0 conda-forge libblas 3.9.0 13_win64_mkl conda-forge libbrotlicommon 1.0.9 h8ffe710_6 conda-forge libbrotlidec 1.0.9 h8ffe710_6 conda-forge libbrotlienc 1.0.9 h8ffe710_6 conda-forge libcblas 3.9.0 13_win64_mkl conda-forge libclang 11.1.0 default_h5c34c98_1 conda-forge libcurl 7.81.0 h789b8ee_0 conda-forge libdeflate 1.8 h8ffe710_0 conda-forge libffi 3.4.2 h8ffe710_5 conda-forge libgdal 3.3.3 h08edb8f_0 conda-forge libglib 2.70.2 h3be07f2_1 conda-forge libiconv 1.16 he774522_0 conda-forge libkml 1.3.0 h9859afa_1014 conda-forge liblapack 3.9.0 13_win64_mkl conda-forge libnetcdf 4.8.1 nompi_h1cc8e9d_101 conda-forge libpng 1.6.37 h1d00b33_2 conda-forge libpq 13.5 hfcc5ef8_1 conda-forge librttopo 1.1.0 hb340de5_6 conda-forge libsodium 1.0.18 h8d14728_1 conda-forge libspatialindex 1.9.3 h39d44d4_4 conda-forge libspatialite 5.0.1 h6a90f17_9 conda-forge libssh2 1.10.0 h680486a_2 conda-forge libtiff 4.3.0 hd413186_2 conda-forge libwebp-base 1.2.2 h8ffe710_0 conda-forge libxml2 2.9.12 hf5bbc77_1 conda-forge libzip 1.8.0 hfed4ece_1 conda-forge libzlib 1.2.11 h8ffe710_1013 conda-forge lz4-c 1.9.3 h8ffe710_1 conda-forge m2w64-gcc-libgfortran 5.3.0 6 conda-forge m2w64-gcc-libs 5.3.0 7 conda-forge m2w64-gcc-libs-core 5.3.0 7 conda-forge m2w64-gmp 6.1.0 2 conda-forge m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markupsafe 2.0.1 py38h294d835_1 conda-forge matplotlib 3.5.1 py38haa244fe_0 conda-forge matplotlib-base 3.5.1 py38h1f000d6_0 conda-forge matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge mistune 0.8.4 py38h294d835_1005 conda-forge mkl 2022.0.0 h0e2418a_796 conda-forge msys2-conda-epoch 20160418 1 conda-forge munch 2.5.0 py_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge mypy_extensions 0.4.3 py38haa244fe_4 conda-forge nbclient 0.5.10 pyhd8ed1ab_1 conda-forge nbconvert 6.4.0 py38haa244fe_0 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge nest-asyncio 1.5.4 pyhd8ed1ab_0 conda-forge networkx 2.6.3 pyhd8ed1ab_1 conda-forge notebook 6.4.7 pyha770c72_0 conda-forge numpy 1.22.0 py38hcf66579_1 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openjpeg 2.4.0 hb211442_1 conda-forge openssl 1.1.1l h8ffe710_0 conda-forge packaging 21.3 pyhd8ed1ab_0 conda-forge pandas 1.3.5 py38h5d928e2_0 conda-forge pandoc 2.17.0.1 h8ffe710_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge pathspec 0.9.0 pyhd8ed1ab_0 conda-forge pcre 8.45 h0e60522_0 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.4.0 py38h794f750_0 conda-forge pip 21.3.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h8ffe710_0 conda-forge platformdirs 2.3.0 pyhd8ed1ab_0 conda-forge poppler 21.09.0 h24fffdf_3 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 13.5 h1c22c4f_1 conda-forge proj 8.1.1 h1cfcee9_2 conda-forge prometheus_client 0.12.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.24 pyha770c72_0 conda-forge prompt_toolkit 3.0.24 hd8ed1ab_0 conda-forge pure_eval 0.2.1 pyhd8ed1ab_1 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pygeos 0.10.2 py38h127f298_0 conda-forge pygments 2.11.2 pyhd8ed1ab_0 conda-forge pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.7 pyhd8ed1ab_0 conda-forge pyproj 3.2.1 py38hfe7211e_2 conda-forge pyqt 5.12.3 py38haa244fe_8 conda-forge pyqt-impl 5.12.3 py38h885f38d_8 conda-forge pyqt5-sip 4.19.18 py38h885f38d_8 conda-forge pyqtchart 5.12 py38h885f38d_8 conda-forge pyqtwebengine 5.12.1 py38h885f38d_8 conda-forge pyrsistent 0.18.1 py38h294d835_0 conda-forge pysocks 1.7.1 py38haa244fe_4 conda-forge python 3.8.12 h7840368_2_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.8 2_cp38 conda-forge pytz 2021.3 pyhd8ed1ab_0 conda-forge pywin32 303 py38h294d835_0 conda-forge pywinpty 1.1.6 py38hd3f51b4_0 conda-forge pyzmq 22.3.0 py38h09162b1_1 conda-forge qt 5.12.9 h5909a2a_4 conda-forge qtconsole 5.2.2 pyhd8ed1ab_1 conda-forge qtconsole-base 5.2.2 pyhd8ed1ab_1 conda-forge qtpy 2.0.0 pyhd8ed1ab_0 conda-forge requests 2.27.1 pyhd8ed1ab_0 conda-forge rtree 0.9.7 py38h8b54edf_3 conda-forge scikit-learn 1.0.2 py38hb60ee80_0 conda-forge scipy 1.7.3 py38ha1292f7_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 59.8.0 py38haa244fe_0 conda-forge shapely 1.8.0 py38h127f298_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.37.0 h8ffe710_0 conda-forge stack_data 0.1.4 pyhd8ed1ab_0 conda-forge tbb 2021.5.0 h2d74725_0 conda-forge terminado 0.12.1 py38haa244fe_1 conda-forge testpath 0.5.0 pyhd8ed1ab_0 conda-forge threadpoolctl 3.0.0 pyh8a188c0_0 conda-forge tiledb 2.3.4 h78dabda_0 conda-forge tk 8.6.11 h8ffe710_1 conda-forge tomli 1.2.2 pyhd8ed1ab_0 conda-forge tornado 6.1 py38h294d835_2 conda-forge traitlets 5.1.1 pyhd8ed1ab_0 conda-forge typed-ast 1.5.1 py38h294d835_0 conda-forge typing_extensions 4.0.1 pyha770c72_0 conda-forge ucrt 10.0.20348.0 h57928b3_0 conda-forge unicodedata2 14.0.0 py38h294d835_0 conda-forge urllib3 1.26.8 pyhd8ed1ab_1 conda-forge vc 14.2 hb210afc_6 conda-forge vs2015_runtime 14.29.30037 h902a5da_6 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge widgetsnbextension 3.5.2 py38haa244fe_1 conda-forge win_inet_pton 1.1.0 py38haa244fe_3 conda-forge winpty 0.4.3 4 conda-forge xerces-c 3.2.3 h0e60522_4 conda-forge xyzservices 2022.1.1 pyhd8ed1ab_0 conda-forge xz 5.2.5 h62dcd97_1 conda-forge zeromq 4.3.4 h0e60522_1 conda-forge zipp 3.7.0 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h8ffe710_1013 conda-forge zstd 1.5.2 h6255e5f_0 conda-forge
dnerini commented 2 years ago

Hello @ErickMesquita and thanks for the update on the issue. Are you running it on windows? Is this happening when pip installing from source (in a conda environemnt) ?

ErickMesquita commented 2 years ago

Hello, yes, my testing machine is an AMDx64 running Windows 7 with conda 4.11.0.

The following code reproduces the error:

conda create --name test-scipy -y -c conda-forge python=3.8 numpy scipy
conda activate test-scipy
python
import scipy

It installs Python 3.8.12, Numpy 1.22.2 and Scipy 1.8.0. Then it throws FileNotFoundError because of that .gfortran-win_amd64.dll file.

I tried Python 3.8.8, 3.8.10 and 3.8.11. I tried Scipy 1.8, 1.7 and 1.6.

( I cannot run Python 3.9+ because it requires Windows 10)