pysal / giddy

Exploratory spatiotemporal data analysis and Geospatial distribution dynamics analysis
http://pysal.org/giddy/
BSD 3-Clause "New" or "Revised" License
76 stars 25 forks source link

np.int depreceated #214

Closed tra6sdc closed 5 months ago

tra6sdc commented 5 months ago

Hello, I got this message :

AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

I am not sure if this is a genuine error message or I am doing something dumb.

My data is panel data 32k rows (areas) by 6 columns (time periods). My input data is pre-processed into quintiles that are NOT pooled over the 6 time periods. Quintiles are calculated within each time period. Thus there are very near 20% of each quintile in each time period. I have used GeoDa to create the .gal file.

I run with the command:

m = giddy.markov.Spatial_Markov(c, w, fixed = False, discrete = True, k = 5 ,m = 5)

I believe that the values I have given to fixed and discrete are the correct ones for my situation. When I do this I get the error message above. The message says something has been deprecated. Thanks.

weikang9009 commented 5 months ago

Thank you for reporting the issue. Could you please provide more details of your program?

tra6sdc commented 5 months ago

Hello, giddy version 2.3.3 and numpy version 1.26.4

This is the entire console Traceback output

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[131], line 1
----> 1 sm = giddy.markov.Spatial_Markov(c,w,fixed=False,discrete=True,k=5,m=5)

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\giddy\markov.py:819, in Spatial_Markov.__init__(self, y, w, k, m, permutations, fixed, discrete, cutoffs, lag_cutoffs, variable_name, fill_empty_classes)
    817 self.p = classic.p
    818 self.transitions = classic.transitions
--> 819 self.T, self.P = self._calc(y, w, fill_empty_classes=fill_empty_classes)
    821 if permutations:
    822     nrp = np.random.permutation

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\giddy\markov.py:941, in Spatial_Markov._calc(self, y, w, fill_empty_classes)
    932 """Helper to estimate spatial lag conditioned Markov transition
    933 probability matrices based on maximum likelihood techniques.
    934 
   (...)
    938 
    939 """
    940 if self.discrete:
--> 941     self.lclass_ids = weights.lag_categorical(w, self.class_ids, ties="tryself")
    942 else:
    943     ly = weights.lag_spatial(w, y)

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\libpysal\weights\spatial_lag.py:163, in lag_categorical(w, y, ties)
    161 if len(orig_shape) > 1:
    162     if orig_shape[1] > 1:
--> 163         return np.vstack([lag_categorical(w, col) for col in y.T]).T
    164 y = y.flatten()
    165 output = np.zeros_like(y)

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\libpysal\weights\spatial_lag.py:163, in <listcomp>(.0)
    161 if len(orig_shape) > 1:
    162     if orig_shape[1] > 1:
--> 163         return np.vstack([lag_categorical(w, col) for col in y.T]).T
    164 y = y.flatten()
    165 output = np.zeros_like(y)

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\libpysal\weights\spatial_lag.py:167, in lag_categorical(w, y, ties)
    165 output = np.zeros_like(y)
    166 labels = np.unique(y)
--> 167 normalized_labels = np.zeros(y.shape, dtype=np.int)
    168 for i, label in enumerate(labels):
    169     normalized_labels[y == label] = i

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\numpy\__init__.py:324, in __getattr__(attr)
    319     warnings.warn(
    320         f"In the future `np.{attr}` will be defined as the "
    321         "corresponding NumPy scalar.", FutureWarning, stacklevel=2)
    323 if attr in __former_attrs__:
--> 324     raise AttributeError(__former_attrs__[attr])
    326 if attr == 'testing':
    327     import numpy.testing as testing

AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
weikang9009 commented 5 months ago

Thanks. There seems to be a version mismatch. The most updated version of giddy is v2.3.5. Could you please try pip install -U giddy and then rerun your program?

tra6sdc commented 5 months ago

Hello, I have done the upgrade (see below) and get the same error. Is my combination of fixed and discrete odd and that's th3 issue?

Requirement already satisfied: giddy in c:\program files\anaconda3\envs\markov_env\lib\site-packages (2.3.3)
Collecting giddy
  Downloading giddy-2.3.5-py3-none-any.whl.metadata (6.4 kB)
Requirement already satisfied: esda>=2.1.1 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from giddy) (2.4.1)
Requirement already satisfied: libpysal>=4.0.1 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from giddy) (4.5.1)
Requirement already satisfied: quantecon>=0.4.7 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from giddy) (0.7.0)
Requirement already satisfied: scipy>=1.3 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from giddy) (1.11.4)
Requirement already satisfied: pandas in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from esda>=2.1.1->giddy) (2.2.1)
Requirement already satisfied: scikit-learn in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from esda>=2.1.1->giddy) (1.4.2)
Requirement already satisfied: beautifulsoup4 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from libpysal>=4.0.1->giddy) (4.12.2)
Requirement already satisfied: jinja2 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from libpysal>=4.0.1->giddy) (3.1.4)
Requirement already satisfied: numpy>=1.3 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from libpysal>=4.0.1->giddy) (1.26.4)
Requirement already satisfied: requests in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from libpysal>=4.0.1->giddy) (2.32.2)
Requirement already satisfied: numba>=0.49.0 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from quantecon>=0.4.7->giddy) (0.59.1)
Requirement already satisfied: sympy in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from quantecon>=0.4.7->giddy) (1.12)
Requirement already satisfied: llvmlite<0.43,>=0.42.0dev0 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from numba>=0.49.0->quantecon>=0.4.7->giddy) (0.42.0)
Requirement already satisfied: soupsieve>1.2 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from beautifulsoup4->libpysal>=4.0.1->giddy) (2.5)
Requirement already satisfied: MarkupSafe>=2.0 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from jinja2->libpysal>=4.0.1->giddy) (2.1.3)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from pandas->esda>=2.1.1->giddy) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from pandas->esda>=2.1.1->giddy) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from pandas->esda>=2.1.1->giddy) (2023.3)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from requests->libpysal>=4.0.1->giddy) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from requests->libpysal>=4.0.1->giddy) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from requests->libpysal>=4.0.1->giddy) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from requests->libpysal>=4.0.1->giddy) (2024.2.2)
Requirement already satisfied: joblib>=1.2.0 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from scikit-learn->esda>=2.1.1->giddy) (1.4.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from scikit-learn->esda>=2.1.1->giddy) (2.2.0)
Requirement already satisfied: mpmath>=0.19 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from sympy->quantecon>=0.4.7->giddy) (1.3.0)
Requirement already satisfied: six>=1.5 in c:\program files\anaconda3\envs\markov_env\lib\site-packages (from python-dateutil>=2.8.2->pandas->esda>=2.1.1->giddy) (1.16.0)
Downloading giddy-2.3.5-py3-none-any.whl (61 kB)
   ---------------------------------------- 0.0/61.1 kB ? eta -:--:--
   ---------------------------------------- 0.0/61.1 kB ? eta -:--:--
   ------ --------------------------------- 10.2/61.1 kB ? eta -:--:--
   -------------------- ------------------- 30.7/61.1 kB 330.3 kB/s eta 0:00:01
   --------------------------------- ------ 51.2/61.1 kB 375.8 kB/s eta 0:00:01
   ---------------------------------------- 61.1/61.1 kB 325.9 kB/s eta 0:00:00
Installing collected packages: giddy
  Attempting uninstall: giddy
    Found existing installation: giddy 2.3.3
    Uninstalling giddy-2.3.3:
      Successfully uninstalled giddy-2.3.3
Successfully installed giddy-2.3.5
weikang9009 commented 5 months ago

I do not think it has anything with your data. Could you please upgrade libpysal as well and rerun your program? pip install -U libpysal

tra6sdc commented 5 months ago

OK. Ta. libpysal is now version 4.10. I re-import giddy and libpysal and I still get the message.

These are my data ....

array([[3, 4, 3, 4, 4, 4],
       [4, 4, 4, 4, 4, 4],
       [4, 5, 5, 5, 5, 5],
       ...,
       [3, 4, 3, 3, 3, 3],
       [4, 4, 3, 4, 3, 1],
       [2, 3, 3, 4, 4, 2]], dtype=int64)

If I remove discrete=True the it runs and produces reasonable outputs. This is why I ask if my data (pre-discretised and not pooled) is at conflict with something.

weikang9009 commented 5 months ago

Thank you for the additional explanation. However, I cannot replicate the error (with discrete=True) on my end. If you could share the complete program and the error message you're encountering, we would be better positioned to help you debug the issue.

The preprocessing of your data (quintile discretization separately for each period) should not create a problem.

tra6sdc commented 5 months ago

Hi, thanks for your engagement. Things have actually got worse for me. Now I can not import giddy, see below. Has the update broken something else?

import numpy as np
import giddy
import libpysal
import matplotlib.pyplot as plt
import mapclassify as mc
import geopandas as gpd
import pandas as pd
import pyogrio
from esda.moran import Moran

with the NEW and DIFFERENT error message:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 2
      1 import numpy as np
----> 2 import giddy
      3 import libpysal
      4 import matplotlib.pyplot as plt

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\giddy\__init__.py:10
      7 import contextlib
      8 from importlib.metadata import PackageNotFoundError, version
---> 10 from . import directional, ergodic, markov, mobility, rank, sequence, util
     12 with contextlib.suppress(PackageNotFoundError):
     13     __version__ = version("giddy")

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\giddy\markov.py:24
     22 import numpy as np
     23 import quantecon as qe
---> 24 from esda.moran import Moran_Local
     25 from libpysal import weights
     26 from scipy import stats

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\esda\__init__.py:12
     10 from .geary_local import Geary_Local
     11 from .geary_local_mv import Geary_Local_MV
---> 12 from .getisord import G, G_Local
     13 from .join_counts import Join_Counts
     14 from .join_counts_local import Join_Counts_Local

File C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\esda\getisord.py:8
      5 __all__ = ["G", "G_Local"]
      7 import warnings
----> 8 from libpysal.common import np, stats
      9 from libpysal.weights.spatial_lag import lag_spatial as slag
     10 from libpysal.weights.util import fill_diagonal

ImportError: cannot import name 'np' from 'libpysal.common' (C:\Program Files\Anaconda3\envs\markov_env\Lib\site-packages\libpysal\common.py)
knaaptime commented 5 months ago

looks like you probably need to update esda as well

tra6sdc commented 5 months ago

OK. I re-started the Notebook, updated esda, and I can now run with discrete=True. Thanks.