pydata / numexpr

Fast numerical array expression evaluator for Python, NumPy, Pandas, PyTables and more
https://numexpr.readthedocs.io/en/latest/user_guide.html
MIT License
2.21k stars 207 forks source link

Complex (imaginary) evaluations #480

Closed sstendahl closed 5 months ago

sstendahl commented 5 months ago

Hey,

We are using numexpr for the Graphs project, and in general it's an amazing module. So really thanks a lot for the efforts on this!

I was looking at how well imaginary components are handled in Graphs (and thus in numexpr which we use for these evaluations), but couldn't really figure out how this works. As a simple minimum example, here's what I've been trying:

value = numexpr.evaluate("sqrt(-4)")  # returns nan
print(value)  # prints nan
imag_value1 = numexpr.evaluate("imag(sqrt(-4))") # returns 0
print(imag_value2)  # prints 0
real_value = numexpr.evaluate("real(sqrt(-4))")  # returns nan
print(real_value)  # prints nan

How would I actually get the imaginary component of sqrt(-4) in this case? Which should be 2. Similarly, the real component should be 0 of course instead of nan.

Not sure if it's a bug, as I feel like I'm just not understanding correctly how complex expressions work with numexpr

27rabbitlt commented 5 months ago

If it’s just about literal number like -4, I think I can quickly add this functionality just modifying python code; if you want it to be able to do smth like: sqrt(x) I ‘m not sure if it’s worth adding an if statement to C code since each time we execute sqrt it will check if it’s a negative number while I guess in the most use cases user will only use sqrt for positive number.

Numexpr supports complex number calculation, so a workaround might be manually write a complex number with 0+sqrt(-x) i

On Thu, Apr 11, 2024 at 11:01 Sjoerd Stendahl @.***> wrote:

Hey,

We are using numexpr for the Graphs https://apps.gnome.org/Graphs/ project, and in general it's an amazing module. So really thanks a lot for the efforts on this!

I was looking at how well imaginary components are handled in Graphs (and thus in numexpr which we use for these evaluations), but couldn't really figure out how this works. As a simple minimum example, here's what I've been trying:

value = numexpr.evaluate("sqrt(-4)") # returns nan print(value) # prints nan imag_value1 = numexpr.evaluate("imag(sqrt(-4))") # returns 0 print(imag_value2) # prints 0 real_value = numexpr.evaluate("real(sqrt(-4))") # returns nan print(real_value) # prints nan

How would I actually get the imaginary component of sqrt(-4) in this case? Which should be 2. Similarly, the reaql component should be 0 of course instead of nan.

Not sure if it's a bug, as I feel like I'm just not understanding correctly how complex expressions work with numexpr

— Reply to this email directly, view it on GitHub https://github.com/pydata/numexpr/issues/480, or unsubscribe https://github.com/notifications/unsubscribe-auth/A33BDH5QEG3E3DXKQ5R3TJLY4ZGO5AVCNFSM6AAAAABGB4YCSWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZTOMRUGY4TENY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

FrancescAlted commented 5 months ago

Casting in numexpr is not as flexible as in numpy, so I'm +1 on @27rabbitlt 's suggestion to manually specify a complex number in your expression.

sstendahl commented 5 months ago

Yeah in this case it's about doing something like sqrt(x), but also in Graphs it would be a niche use-case, and I haven't had any user requests at this point for this functionality either. So I think it makes sense to not complicate the codebase to much for a full implementation. As other functions (e.g. trig functions, and regular powers) can yield imaginary components as well.

So manually adding a complex number would be a perfectly fine solution to me. I am just wondering how that would work, as I couldn't really figure that out from the documentation. Both numexpr.evaluatue("0+sqrt(-x)i"), as well asnumexpr.evaluate("3+i")` yield an error for me:

>>> numexpr.evaluate("3+i")
Traceback (most recent call last):
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 762, in getArguments
    a = local_dict[name]
        ~~~~~~~~~~^^^^^^
KeyError: 'i'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 977, in evaluate
    raise e
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 876, in validate
    arguments = getArguments(names, local_dict, global_dict, _frame_depth=_frame_depth)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 764, in getArguments
    a = global_dict[name]
        ~~~~~~~~~~~^^^^^^
KeyError: 'i'
>>> 
27rabbitlt commented 5 months ago

Yeah in this case it's about doing something like sqrt(x), but also in Graphs it would be a niche use-case, and I haven't had any user requests at this point for this functionality either. So I think it makes sense to not complicate the codebase to much for a full implementation. As other functions (e.g. trig functions, and regular powers) can yield imaginary components as well.

So manually adding a complex number would be a perfectly fine solution to me. I am just wondering how that would work, as I couldn't really figure that out from the documentation. Both numexpr.evaluatue("0+sqrt(-x)i"), as well asnumexpr.evaluate("3+i")` yield an error for me:

>>> numexpr.evaluate("3+i")
Traceback (most recent call last):
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 762, in getArguments
    a = local_dict[name]
        ~~~~~~~~~~^^^^^^
KeyError: 'i'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 977, in evaluate
    raise e
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 876, in validate
    arguments = getArguments(names, local_dict, global_dict, _frame_depth=_frame_depth)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sjoerd/.local/lib/python3.11/site-packages/numexpr/necompiler.py", line 764, in getArguments
    a = global_dict[name]
        ~~~~~~~~~~~^^^^^^
KeyError: 'i'
>>> 

you can try ne.evaluate('1 + 2j'), we use j because that's how Python define complex number literal. Maybe we should support both? @FrancescAlted

FrancescAlted commented 5 months ago

I'd say whatever uses numpy is good for numexpr too ;-)

27rabbitlt commented 5 months ago

This should be closed since it seems already solved. We should stick with j since that's how Python define complex number literals.

sstendahl commented 5 months ago

This should be closed since it seems already solved. We should stick with j since that's how Python define complex number literals.

Apologies for not getting back to this issue earlier, I've had a very busy weekend. Just wanted to confirm that it works with j. I actually tried j before, as that's pretty standard in most Python modules. (As a physicist, not my favourite, but I'm used to it). However, it raised the same KeyError, so I gave up on that.

But I have since found the issue there, and that's simply that it also requires a numerical value to be correctly recognized as an imaginary number. I think the same is true for numpy so I wouldn't consider that a bug, it's probably best to keep the syntax mostly the same as numpy. So in short:

numexpr.evaluate("3 + j") <-- Wrong, will raise KeyError: :"j". numexpr.evaluate("3 + 1j") <-- Correct.

Just thought that be good to note here, in case someone else stumbles upon this issue when googling. Thanks for the help!