pydata / numexpr

Fast numerical array expression evaluator for Python, NumPy, Pandas, PyTables and more
https://numexpr.readthedocs.io/en/latest/user_guide.html
MIT License
2.21k stars 207 forks source link

`NPY_MAXARGS` not a compile time constant #460

Closed seberg closed 9 months ago

seberg commented 9 months ago

I bumped NPY_MAXARGS on NumPy main, however, it seems more useful to make NPY_MAXARGS a runtime constant, rather than a compile time one. So that it gives the correct value for both NumPy 1.x and 2.0 (unlike NPY_MAXDIMS, it is used too much for heap arrays).

This means numexpr needs to use a custom constant for the heap array allocations, but can keep using the operands+1 > NPY_MAXARGS check. In principle it would be nice to have a operands+1 > NUMEXPR_MAXARGS check which maybe could warn (if you don't just remove the need for a compile time constant).

PS: I am open to bumping the constant further and hopefully removing it in the long term. Maybe you can just not rely on it at all anymore, but I am not sure what errors NumPy would raise if you removed it...

27rabbitlt commented 9 months ago

It seems that numexpr should create a new constant which is large enough and replace NPY_MAXARGS where it's used for array size, and keep it where it's used for operands+1 > NUMEXPR_MAXARGS check.

Am I understanding this correctly?

seberg commented 9 months ago

Right, that would work, unless you don't care about doing a normal allocation. Only thing is if NumPy lifts the limit, it would be nice if you just notice, but not sure that there is a good way.

seberg commented 9 months ago

Sounds like this should be fixed, thanks!