EconForge / interpolation.py

BSD 2-Clause "Simplified" License
123 stars 35 forks source link

Numba warnings #110

Closed kp992 closed 4 months ago

kp992 commented 7 months ago
NumbaDeprecationWarning: numba.generated_jit is deprecated. Please see the documentation at: https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-generated-jit for more information and advice on a suitable replacement.
  warnings.warn(msg, NumbaDeprecationWarning)
freol35241 commented 5 months ago

Also this one shows up when using eval_splines:

NumbaPendingDeprecationWarning: Code using Numba extension API maybe depending on 'old_style' error-capturing, which is deprecated and will be replaced by 'new_style' in a future release. See details at https://numba.readthedocs.io/en/latest/reference/deprecation.html#deprecation-of-old-style-numba-captured-errors
Exception origin:
  File "...\interpolation\splines\eval_splines.py", line 106, in __eval_spline
    kk = (order).literal_value
freol35241 commented 5 months ago

And further, generated_jit is now removed as of Numba 0.59.0. This mean useres of interpolaiton.py is unfortunately stuck at Numba < 0.59.0.

mmcky commented 4 months ago

@albop taking a closer look at the code in interpolation/multilinear/fungen.py for example do you see a reason why generated_jit can't be replaced by a direct njit?

I am reading https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-generated-jit

otherwise we can move some of this over to use overload

albop commented 4 months ago

I used to like this function... Anyway @mmcky, last time I touched the library I found a few bugs in numba (https://github.com/numba/numba/issues?q=is%3Aissue+is%3Aopen+albop ), stopped there and pledged to return to it after upstream was fixed. These bugs were not directly related to the use of generated_jit though. I can look into it around the end of next week and make a release which is compatible with latest numba. Reason one cannot use njit is because specific code is generated for specific dimensions combinations of inputs. Yes, overload should work

mmcky commented 4 months ago

@albop @kp992 I have been reading the deprecation notice this morning.

I think this is what is needed for all the generated_jit calls but it seems like a lot of boiler plate.

# returns the index of a 1d point along a 1d dimension (python)
def py_get_index(gc, x):
    if gc == t_coord:
        # regular coordinate
        def fun(gc, x):
            δ = (gc[1] - gc[0]) / (gc[2] - 1)
            d = x - gc[0]
            ii = d // δ
            i = int(ii)
            i = clamp(i, 0, gc[2] - 2)
            r = d - i * δ
            λ = r / δ
            return (i, λ)

        return fun
    else:
        # irregular coordinate
        def fun(gc, x):
            N = gc.shape[0]
            i = int(np.searchsorted(gc, x)) - 1
            i = clamp(i, 0, N - 2)
            λ = (x - gc[i]) / (gc[i + 1] - gc[i])
            return (i, λ)

        return fun

@overload(py_get_index)
def ol_get_index(gc, x):
    if gc == t_coord:
        # regular coordinate
        def fun(gc, x):
            δ = (gc[1] - gc[0]) / (gc[2] - 1)
            d = x - gc[0]
            ii = d // δ
            i = int(ii)
            i = clamp(i, 0, gc[2] - 2)
            r = d - i * δ
            λ = r / δ
            return (i, λ)

        return fun
    else:
        # irregular coordinate
        def fun(gc, x):
            N = gc.shape[0]
            i = int(np.searchsorted(gc, x)) - 1
            i = clamp(i, 0, N - 2)
            λ = (x - gc[i]) / (gc[i + 1] - gc[i])
            return (i, λ)

        return fun

@njit
def get_index(gc, x):
    return py_get_index(gc, x)

rather than the simpler generated_jit. Am I understanding this correctly? I think the types in the python only function should be python types so maybe that should be tuple(float64, float64, int64)) but using python types?

oyamad commented 4 months ago

If get_index will only be called from inside a jitted function, it should be simpler like this (if I remember correctly from QuantEcon/QuantEcon.py#701):

def get_index(gc, x):  # This is called from pure Python
    pass

@overload(get_index)  # This is called from inside a jitted function
def ol_get_index(gc, x):
    if gc == t_coord:
        # regular coordinate
        def fun(gc, x):
            δ = (gc[1] - gc[0]) / (gc[2] - 1)
            d = x - gc[0]
            ii = d // δ
            i = int(ii)
            i = clamp(i, 0, gc[2] - 2)
            r = d - i * δ
            λ = r / δ
            return (i, λ)

        return fun
    else:
        # irregular coordinate
        def fun(gc, x):
            N = gc.shape[0]
            i = int(np.searchsorted(gc, x)) - 1
            i = clamp(i, 0, N - 2)
            λ = (x - gc[i]) / (gc[i + 1] - gc[i])
            return (i, λ)

        return fun
mmcky commented 4 months ago

thanks @oyamad for that link. That is really helpful.

kp992 commented 4 months ago

Thanks @mmcky and @oyamad. I have tried both the approaches, but the tests are failing. I tried to fix interpolation.multilinear first using https://github.com/EconForge/interpolation.py/issues/110#issuecomment-1993520903, but that fails for me.