search

AntSimi / py-eddy-tracker

Eddy identification and tracking
https://py-eddy-tracker.readthedocs.io/en/latest/
GNU General Public License v3.0
133 stars 55 forks source link

ImportError: numpy.core.multiarray failed to import #210

Closed Mrvy29 closed 9 months ago

Mrvy29 commented 1 year ago

Hi Antoine !

I am really enjoying working with this package , thank you ! I've been using it successfully during the last year and have also started using it with google colab.

I've recently encountered a RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf, it seems some dependencies don't work anymore specifically ImportError: numpy.core.multiarray failed to import which never happened before. I tried to solve it independently but haven't managed to get anywhere and would appreciate your help. I want to preface that this issue is only relevent with google colab as it requires new installs and doesn't (to my knowledge) allow packages previously installed to be stored for longer term use.

Code for reproduction

# !pip install pyEddyTracker
#
#

Actual outcome Collecting pyEddyTracker Downloading pyEddyTracker-3.6.1-py3-none-any.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 75.5 MB/s eta 0:00:00 Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (3.7.1) Requirement already satisfied: opencv-python in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (4.8.0.76) Collecting pint==0.18 (from pyEddyTracker) Downloading Pint-0.18-py2.py3-none-any.whl (209 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 kB 21.5 MB/s eta 0:00:00 Collecting polygon3 (from pyEddyTracker) Downloading Polygon3-3.0.9.1.tar.gz (39 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (6.0.1) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (2.31.0) Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (1.10.1) Collecting zarr (from pyEddyTracker) Downloading zarr-2.16.0-py3-none-any.whl (206 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 206.5/206.5 kB 20.9 MB/s eta 0:00:00 Collecting netCDF4<1.6 (from pyEddyTracker) Downloading netCDF4-1.5.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 82.9 MB/s eta 0:00:00 Collecting numpy<1.23 (from pyEddyTracker) Downloading numpy-1.22.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.8/16.8 MB 78.4 MB/s eta 0:00:00 Collecting numba<0.56 (from pyEddyTracker) Downloading numba-0.55.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 88.6 MB/s eta 0:00:00 Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from pint==0.18->pyEddyTracker) (23.1) Collecting cftime (from netCDF4<1.6->pyEddyTracker) Downloading cftime-1.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 56.8 MB/s eta 0:00:00 Collecting llvmlite<0.39,>=0.38.0rc1 (from numba<0.56->pyEddyTracker) Downloading llvmlite-0.38.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.5/34.5 MB 17.0 MB/s eta 0:00:00 Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from numba<0.56->pyEddyTracker) (67.7.2) Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (1.1.0) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (0.11.0) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (4.42.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (1.4.4) Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (9.4.0) Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (3.1.1) Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (2.8.2) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (3.2.0) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (3.4) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (2.0.4) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (2023.7.22) Collecting asciitree (from zarr->pyEddyTracker) Downloading asciitree-0.3.3.tar.gz (4.0 kB) Preparing metadata (setup.py) ... done Collecting fasteners (from zarr->pyEddyTracker) Downloading fasteners-0.18-py3-none-any.whl (18 kB) Collecting numcodecs>=0.10.0 (from zarr->pyEddyTracker) Downloading numcodecs-0.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.7/6.7 MB 36.9 MB/s eta 0:00:00 Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from numcodecs>=0.10.0->zarr->pyEddyTracker) (0.4) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->pyEddyTracker) (1.16.0) Building wheels for collected packages: polygon3, asciitree Building wheel for polygon3 (setup.py) ... done Created wheel for polygon3: filename=Polygon3-3.0.9.1-cp310-cp310-linux_x86_64.whl size=117747 sha256=2ed2a779f7bfd5819c2bf07f2b967ac6b2373f1225504fdde44f1137439f5265 Stored in directory: /root/.cache/pip/wheels/d8/b7/f6/b4e24f56a1cc9856dca98cc2fdc3915d7649b39b62f3dbca9e Building wheel for asciitree (setup.py) ... done Created wheel for asciitree: filename=asciitree-0.3.3-py3-none-any.whl size=5033 sha256=01af4722698a42e396177776cf77bf120def1124a3b502031fa4b5690a78bf34 Stored in directory: /root/.cache/pip/wheels/7f/4e/be/1171b40f43b918087657ec57cf3b81fa1a2e027d8755baa184 Successfully built polygon3 asciitree Installing collected packages: polygon3, asciitree, pint, numpy, llvmlite, fasteners, numcodecs, numba, cftime, zarr, netCDF4, pyEddyTracker Attempting uninstall: numpy Found existing installation: numpy 1.23.5 Uninstalling numpy-1.23.5: Successfully uninstalled numpy-1.23.5 Attempting uninstall: llvmlite Found existing installation: llvmlite 0.39.1 Uninstalling llvmlite-0.39.1: Successfully uninstalled llvmlite-0.39.1 Attempting uninstall: numba Found existing installation: numba 0.56.4 Uninstalling numba-0.56.4: Successfully uninstalled numba-0.56.4 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. plotnine 0.12.2 requires numpy>=1.23.0, but you have numpy 1.22.4 which is incompatible. Successfully installed asciitree-0.3.3 cftime-1.6.2 fasteners-0.18 llvmlite-0.38.1 netCDF4-1.5.8 numba-0.55.2 numcodecs-0.11.0 numpy-1.22.4 pint-0.18 polygon3-3.0.9.1 pyEddyTracker-3.6.1 zarr-2.16.0



`RuntimeError                              Traceback (most recent call last)
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-4-a41439479898>](https://localhost:8080/#) in <cell line: 6>()
      4 
      5 from py_eddy_tracker import data
----> 6 from py_eddy_tracker.dataset.grid import RegularGridDataset

3 frames
[/usr/local/lib/python3.10/dist-packages/Polygon/__init__.py](https://localhost:8080/#) in <module>
      3 
      4 # import everything from cPolygon
----> 5 from Polygon.cPolygon import *
      6 
      7 # keep namespace clean

ImportError: numpy.core.multiarray failed to import`

Really appreciate your support !
Merav
AntSimi commented 1 year ago

It's looks like a numpy version incompatible with polygon

berzinastella commented 1 year ago

Hi, Merav @Mrvy29 , Did you manage to fix your issue? I'm having the same issue and I am wondering if you figured out which numpy is compatible?

Best, Stella

AntSimi commented 1 year ago

Hi, Merav @Mrvy29 , Did you manage to fix your issue? I'm having the same issue and I am wondering if you figured out which numpy is compatible?

Best, Stella

How did you install py eddy tracker?

berzinastella commented 1 year ago

Hi, Merav @Mrvy29 , Did you manage to fix your issue? I'm having the same issue and I am wondering if you figured out which numpy is compatible? Best, Stella

How did you install py eddy tracker?

I got it working by having a 3.9 python version, numpy 1.22.4 and netCDF4 1.6.0, matlpotlib 3.7.1. For higher versions of anything it did not work for me at all. Installed pyeddytracker by pip installing.

AntSimi commented 1 year ago

It's a new environement?

berzinastella commented 1 year ago

It's a new environement?

Yes, i created a new environment for this

AntSimi commented 1 year ago

before to install numpy maybe try to install polygon3

Mrvy29 commented 11 months ago

Hi @berzinastella @AntSimi , I'm sorry it took me so long to get back to this issue. Originally I had resorted to working on my local runtime exclusively because I could not solve the issue. I will however try to install polygon3 before numpy, because in my experience the numba and numpy dependencies were already not compatible hence the py-eddy-tracker wasn't even imported I will update.

Mrvy29 commented 11 months ago

Nope , this hasn't solved this issue. here is my code - just to reiterate , using google Colab , py eddy tracker (that was updated probably) does not work properly because of dependency issues. M

Mrvy29 commented 11 months ago

import pandas as pd import xarray as xr import matplotlib.pyplot as plt import numpy as np !pip install pyEddyTracker import py_eddy_tracker

Collecting pyEddyTracker Downloading pyEddyTracker-3.6.1-py3-none-any.whl (10.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.7/10.7 MB 12.3 MB/s eta 0:00:00 Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (3.7.1) Requirement already satisfied: opencv-python in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (4.8.0.76) Collecting pint==0.18 (from pyEddyTracker) Downloading Pint-0.18-py2.py3-none-any.whl (209 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.8/209.8 kB 25.4 MB/s eta 0:00:00 Collecting polygon3 (from pyEddyTracker) Downloading Polygon3-3.0.9.1.tar.gz (39 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (6.0.1) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (2.31.0) Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from pyEddyTracker) (1.11.4) Collecting zarr (from pyEddyTracker) Downloading zarr-2.16.1-py3-none-any.whl (206 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 206.9/206.9 kB 21.7 MB/s eta 0:00:00 Collecting netCDF4<1.6 (from pyEddyTracker) Downloading netCDF4-1.5.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 61.9 MB/s eta 0:00:00 Collecting numpy<1.23 (from pyEddyTracker) Downloading numpy-1.22.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.8/16.8 MB 56.6 MB/s eta 0:00:00 Collecting numba<0.56 (from pyEddyTracker) Downloading numba-0.55.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 80.6 MB/s eta 0:00:00 Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from pint==0.18->pyEddyTracker) (23.2) Collecting cftime (from netCDF4<1.6->pyEddyTracker) Downloading cftime-1.6.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 65.4 MB/s eta 0:00:00 Collecting llvmlite<0.39,>=0.38.0rc1 (from numba<0.56->pyEddyTracker) Downloading llvmlite-0.38.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.5/34.5 MB 16.7 MB/s eta 0:00:00 Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from numba<0.56->pyEddyTracker) (67.7.2) Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (1.2.0) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (4.46.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (1.4.5) Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (9.4.0) Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (3.1.1) Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->pyEddyTracker) (2.8.2) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (2.0.7) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->pyEddyTracker) (2023.11.17) Collecting asciitree (from zarr->pyEddyTracker) Downloading asciitree-0.3.3.tar.gz (4.0 kB) Preparing metadata (setup.py) ... done Collecting fasteners (from zarr->pyEddyTracker) Downloading fasteners-0.19-py3-none-any.whl (18 kB) Collecting numcodecs>=0.10.0 (from zarr->pyEddyTracker) Downloading numcodecs-0.12.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.7/7.7 MB 76.8 MB/s eta 0:00:00 Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->pyEddyTracker) (1.16.0) Building wheels for collected packages: polygon3, asciitree Building wheel for polygon3 (setup.py) ... done Created wheel for polygon3: filename=Polygon3-3.0.9.1-cp310-cp310-linux_x86_64.whl size=117748 sha256=612766c7b6c994f8b1d738759062720d8a3122eaee53ca95fe2351351f1a159a Stored in directory: /root/.cache/pip/wheels/d8/b7/f6/b4e24f56a1cc9856dca98cc2fdc3915d7649b39b62f3dbca9e Building wheel for asciitree (setup.py) ... done Created wheel for asciitree: filename=asciitree-0.3.3-py3-none-any.whl size=5033 sha256=908eb2fa3f3df91d4ab93f50b283278411f7bacd196e895e634766e2ab372f54 Stored in directory: /root/.cache/pip/wheels/7f/4e/be/1171b40f43b918087657ec57cf3b81fa1a2e027d8755baa184 Successfully built polygon3 asciitree Installing collected packages: polygon3, asciitree, pint, numpy, llvmlite, fasteners, numcodecs, numba, cftime, zarr, netCDF4, pyEddyTracker Attempting uninstall: numpy Found existing installation: numpy 1.23.5 Uninstalling numpy-1.23.5: Successfully uninstalled numpy-1.23.5 Attempting uninstall: llvmlite Found existing installation: llvmlite 0.41.1 Uninstalling llvmlite-0.41.1: Successfully uninstalled llvmlite-0.41.1 Attempting uninstall: numba Found existing installation: numba 0.58.1 Uninstalling numba-0.58.1: Successfully uninstalled numba-0.58.1 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lida 0.0.10 requires fastapi, which is not installed. lida 0.0.10 requires kaleido, which is not installed. lida 0.0.10 requires python-multipart, which is not installed. lida 0.0.10 requires uvicorn, which is not installed. plotnine 0.12.4 requires numpy>=1.23.0, but you have numpy 1.22.4 which is incompatible. tensorflow 2.14.0 requires numpy>=1.23.5, but you have numpy 1.22.4 which is incompatible. Successfully installed asciitree-0.3.3 cftime-1.6.3 fasteners-0.19 llvmlite-0.38.1 netCDF4-1.5.8 numba-0.55.2 numcodecs-0.12.1 numpy-1.22.4 pint-0.18 polygon3-3.0.9.1 pyEddyTracker-3.6.1 zarr-2.16.1 WARNING: The following packages were previously imported in this runtime: [numpy] You must restart the runtime in order to use newly installed versions.

Mrvy29 commented 11 months ago

An update about this issue, using brew and poetry and a pyenv, using python 3.10.13 made the imports all possible without raising API error or grid issues. Nonetheless eddy detection doesn't work properly using the final : a, c = g.eddy_identification("adt", "u", "v", date, 0.002, shape_error=55) see issue #227

AntSimi commented 9 months ago

Related to #217