search

pysal / esda

statistics and classes for exploratory spatial data analysis
https://pysal.org/esda
BSD 3-Clause "New" or "Revised" License
206 stars 53 forks source link

Local G Autocorrelation causing Numba TypingError #275

Open habr-ioki opened 8 months ago

habr-ioki commented 8 months ago

Hey, I'm getting following error when calling esda.getisord.G_Local(df["column"], w).

Input params are of type

As far as I understand the input types are correct and I also don't get any dimension mismatch errors.

Unfortunately I don't really understand the error message, but it seems to me that there's sth. not working as expected with numba (numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)).

Can anyone help me with this issue?

Python version: 3.11 libpysal: 4.9.2 pysal: 23.7 esda: 2.5.1

---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py", line 468, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<intrinsic _impl>) found for signature:

 >>> _impl(array(int64, 2d, C), array(float64, 1d, C))

There are 2 candidate implementations:
  - Of which 2 did not match due to:
  Intrinsic in function 'dot_2_impl.<locals>._impl': File: numba/np/linalg.py: Line 543.
    With argument(s): '(array(int64, 2d, C), array(float64, 1d, C))':
   Rejected as the implementation raised a specific error:
     TypingError: '@' arguments must all have the same dtype
  raised from [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py:563](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py:563)

During: resolving callee type: Function(<intrinsic _impl>)
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py) (582)

File "../../../.venv/lib/python3.11/site-packages/numba/np/linalg.py", line 582:
            def _dot2_codegen(context, builder, sig, args):
                <source elided>

        return lambda left, right: _impl(left, right)
        ^

During: typing of intrinsic-call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py) (650)

File "../../../.venv/lib/python3.11/site-packages/esda/getisord.py", line 650:
def _g_local_crand(i, z, permuted_ids, weights_i, scaling):
    <source elided>
    zi, zrand = _prepare_univariate(i, z, permuted_ids, other_weights)
    return (zrand @ other_weights) [/](https://file+.vscode-resource.vscode-cdn.net/) (scaling - zi)
    ^

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

File "../../../.venv/lib/python3.11/site-packages/esda/crand.py", line 328:
def compute_chunk(
    <source elided>
        mask[chunk_start + i] = False
        rstats = stat_func(chunk_start + i, z, permuted_ids, weights_i, scaling)
        ^

"""

The above exception was the direct cause of the following exception:

TypingError                               Traceback (most recent call last)
[/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb) Cell 22 line 2
      [1](vscode-notebook-cell:/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb#Y103sZmlsZQ%3D%3D?line=0) # Gi
----> [2](vscode-notebook-cell:/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb#Y103sZmlsZQ%3D%3D?line=1) go_i = esda.getisord.G_Local(uni_grid_gdf["potential_index"], w)
      [3](vscode-notebook-cell:/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb#Y103sZmlsZQ%3D%3D?line=2) #go_i
      [4](vscode-notebook-cell:/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb#Y103sZmlsZQ%3D%3D?line=3) # Gi*
      [5](vscode-notebook-cell:/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/regionalization.ipynb#Y103sZmlsZQ%3D%3D?line=4) #go_i_star = esda.getisord.G_Local(uni_grid_gdf["potential_index"].to_numpy(), w, star=True)

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:426](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:426), in G_Local.__init__(self, y, w, transform, permutations, star, keep_simulations, n_jobs, seed, island_weight)
    [424](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:424) self.p_norm = stats.norm.sf(np.abs(self.Zs))
    [425](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:425) if permutations:
--> [426](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:426)     self.p_sim, self.rGs = _crand_plus(
    [427](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:427)         y,
    [428](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:428)         w,
    [429](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:429)         self.Gs,
    [430](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:430)         permutations,
    [431](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:431)         keep_simulations,
    [432](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:432)         n_jobs=n_jobs,
    [433](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:433)         stat_func=_g_local_star_crand if star else _g_local_crand,
    [434](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:434)         scaling=y.sum(),
    [435](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:435)         seed=seed,
    [436](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:436)         island_weight=island_weight,
    [437](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:437)     )
    [438](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:438)     if keep_simulations:
    [439](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py:439)         self.sim = sim = self.rGs.T

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:208](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:208), in crand(z, w, observed, permutations, keep, n_jobs, stat_func, scaling, seed, island_weight)
    [206](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:206)         n_jobs = len(z)
    [207](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:207)     # Parallel implementation
--> [208](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:208)     larger, rlocals = parallel_crand(
    [209](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:209)         z,
    [210](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:210)         observed,
    [211](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:211)         cardinalities,
    [212](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:212)         self_weights,
    [213](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:213)         other_weights,
    [214](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:214)         permuted_ids,
    [215](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:215)         scaling,
    [216](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:216)         n_jobs,
    [217](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:217)         keep,
    [218](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:218)         stat_func,
    [219](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:219)         island_weight,
    [220](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:220)     )
    [222](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:222) low_extreme = (permutations - larger) < larger
    [223](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:223) larger[low_extreme] = permutations - larger[low_extreme]

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:545](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:545), in parallel_crand(z, observed, cardinalities, self_weights, other_weights, permuted_ids, scaling, n_jobs, keep, stat_func, island_weight)
    [533](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:533) chunks = chunk_generator(
    [534](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:534)     n_jobs,
    [535](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:535)     starts,
ref='~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:0'>0</a>;32m   (...)
    [541](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:541)     w_boundary_points,
    [542](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:542) )
    [544](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:544) with parallel_backend("loky", inner_max_num_threads=1):
--> [545](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:545)     worker_out = Parallel(n_jobs=n_jobs)(
    [546](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:546)         delayed(compute_chunk)(
    [547](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:547)             *pars, permuted_ids, scaling, keep, stat_func, island_weight
    [548](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:548)         )
    [549](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:549)         for pars in chunks
    [550](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:550)     )
    [551](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:551) larger, rlocals = zip(*worker_out)
    [552](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py:552) larger = np.hstack(larger).squeeze()

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1952](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1952), in Parallel.__call__(self, iterable)
   [1946](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1946) # The first item from the output is blank, but it makes the interpreter
   [1947](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1947) # progress until it enters the Try/Except block of the generator and
   [1948](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1948) # reach the first `yield` statement. This starts the aynchronous
   [1949](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1949) # dispatch of the tasks to the workers.
   [1950](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1950) next(output)
-> [1952](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1952) return output if self.return_generator else list(output)

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1595](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1595), in Parallel._get_outputs(self, iterator, pre_dispatch)
   [1592](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1592)     yield
   [1594](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1594)     with self._backend.retrieval_context():
-> [1595](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1595)         yield from self._retrieve()
   [1597](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1597) except GeneratorExit:
   [1598](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1598)     # The generator has been garbage collected before being fully
   [1599](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1599)     # consumed. This aborts the remaining tasks if possible and warn
   [1600](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1600)     # the user if necessary.
   [1601](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1601)     self._exception = True

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1699](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1699), in Parallel._retrieve(self)
   [1692](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1692) while self._wait_retrieval():
   [1693](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1693) 
   [1694](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1694)     # If the callback thread of a worker has signaled that its task
   [1695](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1695)     # triggered an exception, or if the retrieval loop has raised an
   [1696](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1696)     # exception (e.g. `GeneratorExit`), exit the loop and surface the
   [1697](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1697)     # worker traceback.
   [1698](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1698)     if self._aborting:
-> [1699](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1699)         self._raise_error_fast()
   [1700](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1700)         break
   [1702](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1702)     # If the next job is not ready for retrieval yet, we just wait for
   [1703](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1703)     # async callbacks to progress.

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1734](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1734), in Parallel._raise_error_fast(self)
   [1730](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1730) # If this error job exists, immediatly raise the error by
   [1731](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1731) # calling get_result. This job might not exists if abort has been
   [1732](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1732) # called directly or if the generator is gc'ed.
   [1733](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1733) if error_job is not None:
-> [1734](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:1734)     error_job.get_result(self.timeout)

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:736](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:736), in BatchCompletionCallBack.get_result(self, timeout)
    [730](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:730) backend = self.parallel._backend
    [732](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:732) if backend.supports_retrieve_callback:
    [733](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:733)     # We assume that the result has already been retrieved by the
    [734](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:734)     # callback thread, and is stored internally. It's just waiting to
    [735](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:735)     # be returned.
--> [736](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:736)     return self._return_or_raise()
    [738](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:738) # For other backends, the main thread needs to run the retrieval step.
    [739](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:739) try:

File [~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:754](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:754), in BatchCompletionCallBack._return_or_raise(self)
    [752](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:752) try:
    [753](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:753)     if self.status == TASK_ERROR:
--> [754](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:754)         raise self._result
    [755](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:755)     return self._result
    [756](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/data_analytics/areas_of_high_potential/notebooks/~/Documents/data-analytics/.venv/lib/python3.11/site-packages/joblib/parallel.py:756) finally:

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<intrinsic _impl>) found for signature:

 >>> _impl(array(int64, 2d, C), array(float64, 1d, C))

There are 2 candidate implementations:
  - Of which 2 did not match due to:
  Intrinsic in function 'dot_2_impl.<locals>._impl': File: numba/np/linalg.py: Line 543.
    With argument(s): '(array(int64, 2d, C), array(float64, 1d, C))':
   Rejected as the implementation raised a specific error:
     TypingError: '@' arguments must all have the same dtype
  raised from [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py:563](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py:563)

During: resolving callee type: Function(<intrinsic _impl>)
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/numba/np/linalg.py) (582)

File "../../../.venv/lib/python3.11/site-packages/numba/np/linalg.py", line 582:
            def _dot2_codegen(context, builder, sig, args):
                <source elided>

        return lambda left, right: _impl(left, right)
        ^

During: typing of intrinsic-call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/getisord.py) (650)

File "../../../.venv/lib/python3.11/site-packages/esda/getisord.py", line 650:
def _g_local_crand(i, z, permuted_ids, weights_i, scaling):
    <source elided>
    zi, zrand = _prepare_univariate(i, z, permuted_ids, other_weights)
    return (zrand @ other_weights) [/](https://file+.vscode-resource.vscode-cdn.net/) (scaling - zi)
    ^

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

During: resolving callee type: type(CPUDispatcher(<function _g_local_crand at 0x107b968e0>))
During: typing of call at [/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py](https://file+.vscode-resource.vscode-cdn.net/Users/hb/Documents/data-analytics/.venv/lib/python3.11/site-packages/esda/crand.py) (328)

File "../../../.venv/lib/python3.11/site-packages/esda/crand.py", line 328:
def compute_chunk(
    <source elided>
        mask[chunk_start + i] = False
        rstats = stat_func(chunk_start + i, z, permuted_ids, weights_i, scaling)
        ^
ljwolf commented 8 months ago

Interesting... is the dtype of your input series int64? Could you try casting it to float64?

habr-ioki commented 7 months ago

Amazing, that was the reason for this error. Thank you for this hint! Now it's working as expected.

May I suggest referring to this in the esda.G_Local docu? It only says that the y parameter should be of type array. Perhaps you could specify this as an array of floats.

martinfleis commented 7 months ago

I think that we should check the dtype and cast it ourselves under the hood if it comes as int.

ljwolf commented 7 months ago

Indeed, we should do this casting to float64 ourselves for all statistics that hit the conditional inference engine