mdbartos / pysheds

:earth_americas: Simple and fast watershed delineation in python.
GNU General Public License v3.0
702 stars 188 forks source link

Re-evaluate Numba performance #229

Open groutr opened 9 months ago

groutr commented 9 months ago

I think the usage of numba for certain functions needs to be re-evaluated. Sometimes the numba version is much slower than just doing things in numpy.

Example:

def np_vec(affine, x, y):
    a, b, c, d, e, f, _, _, _ = affine
    new_x = x * a + y * b + c
    new_y = x * d + y * e + f
    return new_x, new_y
x = np.random.rand(100)
y = np.random.rand(100)

%timeit _affine_map_vec_numba((1, 2, 3, 4, 5, 6, 7, 8, 9), x, y)
82.8 µs ± 13 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

%timeit np_vec((1, 2, 3, 4, 5, 6, 7, 8, 9), x, y)
17.4 µs ± 1.04 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
mdbartos commented 8 months ago

Is this related to #230?

groutr commented 8 months ago

@mdbartos It is tangentially related. #230 addresses the issue of numba being less efficient due to the strict typing. This ticket is to address the set of cases where numpy is just simply faster than using numba.

The case presented above is the code for a scalar function. The array case uses numba, but for such a simple computation, using numpy produces a faster result. Numexpr is a library that could be of use here to parallelize the computation in a memory efficient way.