&& and || do not short-circuit in macros

[alihaddad-dynideas]

Hello All,

Thanks for your support on JuMP. I have an issue in JuMP that only happens with Mac OS and specific JuMP versions. My mac is 14.2.1 Julia version is 1.9.4, JuMP any version higher than 1.15, Gurobi 1.2.0.

When I run this script I get the following error. This only happens on Mac, if I run it in Ubuntu, I don't see this problem, or if I change the JuMP to any version lower than 1.10, I also don't see the error:

using JuMP, Gurobi
gurobiEnv  = Gurobi.Env()
model = Model(optimizer_with_attributes(() -> Gurobi.Optimizer(gurobiEnv), "OutputFlag" => 1));
dummyKeys= Dict()
dummyKeys[(102,89596)] = 1
@constraint(model, [c in  [102], p in [61463, 59745, 89596]],
    1>= sum( 1 for i in [c] if (c,p) in keys(dummyKeys) && dummyKeys[c,p] == 1 )
)

ERROR: KeyError: key (102, 61463) not found
Stacktrace:
  [1] getindex
    @ ./dict.jl:484 [inlined]
  [2] getindex(::Dict{Any, Any}, ::Int64, ::Int64)
    @ Base ./abstractdict.jl:549
  [3] macro expansion
    @ ~/.julia/packages/MutableArithmetics/NIXlP/src/rewrite_generic.jl:321 [inlined]
  [4] macro expansion
    @ ~/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:321 [inlined]
  [5] macro expansion
    @ ~/.julia/packages/JuMP/OUdu2/src/macros.jl:710 [inlined]
  [6] (::var"#5#6"{Model})(c::Int64, p::Int64)
    @ Main ~/.julia/packages/JuMP/OUdu2/src/Containers/macro.jl:301
  [7] #87
    @ ~/.julia/packages/JuMP/OUdu2/src/Containers/container.jl:124 [inlined]
  [8] iterate
    @ ./generator.jl:47 [inlined]
  [9] collect(itr::Base.Generator{JuMP.Containers.VectorizedProductIterator{Tuple{Vector{Int64}, Vector{Int64}}}, JuMP.Containers.var"#87#89"{var"#5#6"{Model}}})
    @ Base ./array.jl:782
 [10] map(f::Function, A::JuMP.Containers.VectorizedProductIterator{Tuple{Vector{Int64}, Vector{Int64}}})
    @ Base ./abstractarray.jl:3291
 [11] container(f::Function, indices::JuMP.Containers.VectorizedProductIterator{Tuple{Vector{Int64}, Vector{Int64}}}, ::Type{JuMP.Containers.DenseAxisArray}, names::Vector{Any})
    @ JuMP.Containers ~/.julia/packages/JuMP/OUdu2/src/Containers/container.jl:123
 [12] container(f::Function, indices::JuMP.Containers.VectorizedProductIterator{Tuple{Vector{Int64}, Vector{Int64}}}, #unused#::Type{JuMP.Containers.AutoContainerType}, names::Vector{Any})
    @ JuMP.Containers ~/.julia/packages/JuMP/OUdu2/src/Containers/container.jl:75
 [13] macro expansion
    @ ~/.julia/packages/JuMP/OUdu2/src/macros.jl:1211 [inlined]
 [14] top-level scope
    @ REPL[7]:1

[odow]

Thanks for the report. I've reproduced this down to:

julia> using JuMP

julia> model = Model();

julia> data = Dict(2 => 1);

julia> @expression(model, sum(1 for p in [1, 2] if p in keys(data) && data[p] == 1))
ERROR: KeyError: key 1 not found
Stacktrace:
 [1] getindex(h::Dict{Int64, Int64}, key::Int64)
   @ Base ./dict.jl:498
 [2] macro expansion
   @ ~/.julia/packages/MutableArithmetics/NIXlP/src/rewrite_generic.jl:321 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:321 [inlined]
 [4] macro expansion
   @ ~/.julia/packages/JuMP/027Gt/src/macros.jl:239 [inlined]
 [5] macro expansion
   @ ~/.julia/packages/JuMP/027Gt/src/macros/@expression.jl:86 [inlined]
 [6] macro expansion
   @ ~/.julia/packages/JuMP/027Gt/src/macros.jl:375 [inlined]
 [7] top-level scope
   @ ./REPL[54]:1

The issue is because we're not short-circuiting the && operator:

julia> @macroexpand @expression(model, sum(1 for p in [1, 2] if p in keys(data) && data[p] == 1))
quote
    #= REPL[55]:1 =#
    JuMP._valid_model(model, :model)
    begin
        #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros.jl:374 =#
        let model = model
            #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros.jl:375 =#
            begin
                #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros/@expression.jl:86 =#
                begin
                    #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros.jl:239 =#
                    var"#815###431" = begin
                            #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:325 =#
                            let
                                #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:326 =#
                                begin
                                    #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:321 =#
                                    var"#816###432" = MutableArithmetics.Zero()
                                    for p = [1, 2]
                                        #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite_generic.jl:321 =#

# Problem is this next line. Not short circuiting

                                        if (NonlinearOperator(&, :&&))(p in keys(data), (NonlinearOperator(==, :==))(data[p], 1))
                                            #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite_generic.jl:322 =#
                                            begin
                                                #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite_generic.jl:268 =#
                                                var"#816###432" = (MutableArithmetics.operate!!)(MutableArithmetics.add_mul, var"#816###432", 1)
                                            end
                                        end
                                    end
                                end
                                #= /Users/oscar/.julia/packages/MutableArithmetics/NIXlP/src/rewrite.jl:327 =#
                                var"#816###432"
                            end
                        end
                    #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros.jl:240 =#
                    var"#817###433" = (JuMP.flatten!)(var"#815###431")
                end
                #= /Users/oscar/.julia/packages/JuMP/027Gt/src/macros/@expression.jl:89 =#
                JuMP._replace_zero(model, var"#817###433")
            end
        end
    end
end

Let me see if we can fix this.

[alihaddad-dynideas]

Thank you Oscar!

[odow]

A short-term work-around is

using JuMP
model = Model();
data = Dict(2 => 1);
check(data, p) = p in keys(data) && data[p] == 1
@expression(model, sum(1 for p in [1, 2] if check(data, p)))