Running 0.9.7 model in 0.9.8: AttributeError: 'numpy.bool_' object has no attribute 'is_bool' (reification related?)

[hakank]

The following old v0.9.7 model throws AttributeError: 'numpy.bool_' object has no attribute 'is_bool') when running in v9.7.8. (It's my model http://hakank.org/cpmpy/picking_teams.py )

import sys,math,random
import numpy as np
from cpmpy import *
from cpmpy.solvers import *

def picking_teams(s):
    n = len(s)
    n2 = math.ceil(n / 2)

    s_sum = sum(s)
    count = [n2,n2]

    print("s_sum:",s_sum)
    print("s_sum % 2:", s_sum % 2)

    x = intvar(1,2,shape=n,name="x")

    # the difference in strength between the teams
    # to be minimized
    d = intvar(0,s_sum,name="d")

    model = Model([d == abs(sum([s[i]*(x[i] == 1) for i in range(n)]) -
                            sum([s[i]*(x[i] == 2) for i in range(n)])
                            ),
                   ],
                 minimize=d
                  )

    # same size of team
    model += [sum([x[i] == 1  for i in range(n)]) == n2,
              sum([x[i] == 2  for i in range(n)]) == n2]

    # symmetry breaking: assign first person to team 1
    model += (x[0] == 1)

    # divisibility of the sum
    model += ((s_sum % 2) == (d % 2))

    num_solutions = 0
    ss = CPM_ortools(model)

    if ss.solve() is not False:
        num_solutions += 1
        xval = x.value()
        print("x:", xval)        
        print("d:", d.value())
        print("Team 1:", [i for i in range(n) if xval[i] == 1])
        print("Team 2:", [i for i in range(n) if xval[i] == 2])
        print(flush=True)
    print()
    print("number of solutions:", num_solutions)
    print("Num conflicts:", ss.ort_solver.NumConflicts())
    print("NumBranches:", ss.ort_solver.NumBranches())
    print("WallTime:", ss.ort_solver.WallTime())
    print()

s = [35, 52, 17, 26, 90, 55, 57, 54, 41, 9, 75, 24, 17, 23, 62, 74, 100, 67, 40, 48, 7, 6, 44, 19, 16, 14, 2, 66, 70, 2, 43, 45, 76, 53, 90, 12, 88, 96, 30, 30, 36, 93, 74, 1, 52, 45, 38, 7, 24, 96, 17, 21, 12, 12, 23, 90, 77, 64, 37, 79, 67, 62, 24, 11, 74, 82, 51, 17, 72, 18, 37, 94, 43, 44, 32, 86, 94, 33, 97, 27, 38, 38, 29, 92, 35, 82, 22, 66, 80, 8, 62, 72, 25, 13, 94, 42, 51, 31, 69, 66]
picking_teams(s)

# Randomize s
n = 1000
s = [random.randint(1,100) for _ in range(n)]
picking_teams(s)

The output running the model:

s_sum: 4753
s_sum % 2: 1
Traceback (most recent call last):
  File "picking_teams.py", line 96, in <module>
    picking_teams(s)
  File "picking_teams.py", line 73, in picking_teams
    ss = CPM_ortools(model)
  File "/usr/local/lib/python3.7/site-packages/cpmpy/solvers/ortools.py", line 91, in __init__
    super().__init__(name="ortools", cpm_model=cpm_model)
  File "/usr/local/lib/python3.7/site-packages/cpmpy/solvers/solver_interface.py", line 83, in __init__
    self += cpm_model.constraints
  File "/usr/local/lib/python3.7/site-packages/cpmpy/solvers/ortools.py", line 310, in __add__
    cpm_cons = only_bv_implies(flatten_constraint(cpm_con))
  File "/usr/local/lib/python3.7/site-packages/cpmpy/transformations/flatten_model.py", line 136, in flatten_constraint
    flatcons = [flatten_constraint(e) for e in expr]
  File "/usr/local/lib/python3.7/site-packages/cpmpy/transformations/flatten_model.py", line 136, in <listcomp>
    flatcons = [flatten_constraint(e) for e in expr]
  File "/usr/local/lib/python3.7/site-packages/cpmpy/transformations/flatten_model.py", line 143, in flatten_constraint
    assert expr.is_bool(), f"Boolean expressions only in flatten_constraint, `{expr}` not allowed."
AttributeError: 'numpy.bool_' object has no attribute 'is_bool'

Here are some other models that throws the same error:

• http://hakank.org/cpmpy/bus_scheduling_csplib.py (data files in http://hakank.org/cpmpy/bus_scheduling_csplib/)

• http://hakank.org/cpmpy/linear_combinations.py

• http://hakank.org/cpmpy/partition_into_subsets_of_equal_values.py

For all these models, the error involves this line

...
cpm_cons = only_bv_implies(flatten_constraint(cpm_con))
...

so it seems that there is some issue with the reification. Has the syntax/logic for reification been changed?

[IgnaceBleukx]

Hi Hakan, After some digging I found out this line is what causing the trouble: model += ((s_sum % 2) == (d % 2)). In particular, the left hand side gets evaluated to a numpy int64 object. Because the numpy int is on the left hand side of the comparison, the __eq__ method of numpy gets called instead of our overloaded methods. As such, the expression evaluates to np._True instead of CPMpy expression.

To fix this, you can simply swap the order of the arguments so the CPMpy expression comes first, or cast the left hand side to a Python-native int.

Regarding the line you mentioned, these are the transformations we invoke when solving using OR-Tools.

Kind regards, Ignace

[hakank]

Thanks for identifying the problem, Ignace.

For picking_teams.py and partition_into_subsets_of_equal_values.py it was solved by either of your methods (move constant to RHS or cast to int).

For some reason, bus_scheduling_csplib.py works when updating to master version instead of v0.9.8.

However, for linear_combinations.py I could not find the culprit. Also, it now (in master version) throws another error TypeError: The numpy boolean negative, the-operator, is not supported, use the~operator or the logical_not function instead.. I'll check this more and report a new issue if I cannot fix the problem.

[hakank]

And now I found the problem in linear_combinations.py, It was this constraint

  # ...
 for i in range(1,n1):
    for j in range(i+1,n1):
      model += [ (x[i] + x[j] == 2) <= (sum([d==abs(i-j) for d in ds]) > 0) ]

It was fixed by casting the sum constraint to an int:

      model += [ (x[i] + x[j] == 2) <= sum([d==abs(i-j) for d in ds]) > 0) ]

I hope that you regard this behavior as a bug, since having to cast (some) constants or expressions to ints (or move to RHS) is definitely a gotcha, especially since the error messages are not very clear.

[tias]

Smaller example:

1 == d % 2          # Out: (d) mod 2 == 1
np.int_(1) == d % 2 # Out: True
d % 2 == np.int_(1) # Out: (d) mod 2 == 1

In case of a python int, it gives precedence to the `eq' of the CPMpy expression.

But in case of numpy arrays, both are 'custom' objects so the first one (of numpy) is chosen...

But is it a caveat or a bug... the latter would mean there exists a way to fix it, and that I don't know yet.

It seems that numpy actually has a 'one-place only hierarchy' for handing __eq__ and other comparisons/operators, namely ufuncs: https://numpy.org/doc/stable/reference/arrays.classes.html#numpy.class.__array_ufunc__

I do not fully grasp all of it yet, but it seems their might be a fix for this issue, perhaps if we have CPMpy expressions subclass NDArrayOperatorsMixin and implement our intended behaviour through ufuncs.

That page also suggests that for our cpmpy arrays (that subclass ndarray) we should perhaps not overwrite __eq__ but use this array-ufunc funk too.

This is to be looked at more properly at some point.

It is definitely not a feature.