The truth value of a Series is ambiguous

[nursimaaigoritma]

Hello,

I want to compare pandas Series, like

import asteval
import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.randint(0, 10, size=(10, 2)),
    columns=["a", "b"]
)

interpreter = asteval.Interpreter(
            use_numpy=True,
            user_symbols={
                'pd': pd,
                'np': np,
                'df': df
            }
)

interpreter.eval("df.a > df.b")

print(df.a > df.b)

But it gives me this error

ValueError
   df.a > df.b
The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

The line print(df.a > df.b) works fine with the output:

0     True
1    False
2    False
3     True
4    False
5    False
6    False
7    False
8    False
9     True
dtype: bool

Is there something I'm doing wrong? Thank you.

[newville]

@nursimaaigoritma Sorry for the trouble. I think I understand and can fix the problem. The "handle comparison(s) operator" code (which is what a > b uses) also handles multiple chained comparisons (a > b > c), and we want a way to check for False values as we go to short-circuit unneeded tests. So for each, we check whether we can just test the truth value of the result. But: there is an explicit check for numpy arrays, which will also have an array of bools and we do not check the truth value of these. Of course, pandas Dataframes are not numpy arrays, so that test of "should we not try to short-circuit this value" fails.

I think the right solution is not to check whether it is a numpy array (it will miss other sequence objects too), but to just try to check whether the value so far agrees with "not True", and break then.

I hope to push a fix to GitHub today. If you're OK with installing from GH master branch, I may not push a release for this immediately...

[newville]

@nursimaaigoritma I think this is now fixed in the master branch, though I have not added a test for this case yet.

[nursimaaigoritma]

Thank you so much! I'll try getting it from the master branch.