Stackoverflow during Clarabel.setup!

[StoffelOdw]

While executing

Clarabel.setup!(solver, P, q, ccA, ccb, cones, settings)

is get a stackoverflow error:

StackOverflowError:

Stacktrace:
     [1] Array
       @ ./boot.jl:477 [inlined]
     [2] Array
       @ ./boot.jl:486 [inlined]
     [3] zeros
       @ ./array.jl:636 [inlined]
     [4] zeros
       @ ./array.jl:632 [inlined]
     [5] Dict{Symbol, Union{}}()
       @ Base ./dict.jl:70
     [6] Dict{Symbol, Union{}}(kv::@Kwargs{})
       @ Base ./dict.jl:81
     [7] dict_with_eltype
       @ ./abstractdict.jl:580 [inlined]
     [8] Dict(kv::@Kwargs{})
       @ Base ./dict.jl:109
     [9] setup!(s::Clarabel.Solver{Float64}, P::SparseMatrixCSC{Float64, Int64}, c::Matrix{Float64}, A::SparseMatrixCSC{Float64, Int64}, b::SparseVector{Float64, Int64}, cones::Vector{Clarabel.SupportedCone}; kwargs::@Kwargs{})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:71
    [10] setup!(s::Clarabel.Solver{Float64}, P::SparseMatrixCSC{Float64, Int64}, c::Matrix{Float64}, A::SparseMatrixCSC{Float64, Int64}, b::SparseVector{Float64, Int64}, cones::Vector{Clarabel.SupportedCone})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:69
    [11] setup!(s::Clarabel.Solver{Float64}, P::SparseMatrixCSC{Float64, Int64}, c::Matrix{Float64}, A::SparseMatrixCSC{Float64, Int64}, b::SparseVector{Float64, Int64}, cones::Vector{Clarabel.SupportedCone}; kwargs::@Kwargs{})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:72
--- the last 2 lines are repeated 39986 more times ---
 [79984] setup!(s::Clarabel.Solver{Float64}, P::SparseMatrixCSC{Float64, Int64}, c::Matrix{Float64}, A::SparseMatrixCSC{Float64, Int64}, b::SparseVector{Float64, Int64}, cones::Vector{Clarabel.SupportedCone})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:69

The parameter to setupare as follows:

P = sparse(zeros(3,3))

f:
1×3 Matrix{Float64}:
 0.0  0.0  -1.0

ccA:
7×3 SparseMatrixCSC{Float64, Int64} with 11 stored entries:
  1.0          ⋅         ⋅ 
   ⋅          1.0        ⋅ 
  0.015       0.05      1.0
   ⋅           ⋅         ⋅ 
  0.303653    0.443741   ⋅ 
 -0.240127    0.151639   ⋅ 
  0.0115516  -0.721179   ⋅ 

ccb:
7-element Vector{Float64}:
  0.9
  0.3
  0.02
  0.4
  0.42902774162573265
  0.12537428392422256
 -0.014713457989615375

cones = [Clarabel.NonnegativeConeT(length(2)), Clarabel.SecondOrderConeT(3)]

settings = Clarabel.Settings(max_iter = 15, verbose = true);
solver   = Clarabel.Solver();

I have no idea, what caused this stackoverflow.

Can you please help me?

Thank you Stoffel

P.S.: Running Julia 1.10.4 and Clarabel v0.9.0

[mipals]

Hi Stoffel,

My first question: What is q? Maybe it is f from your parameters? If f is q then you should define it as a column vector. (Using f as q gives me an overflow error that resembles the one you state. This is fixed by making f a column vector). My second questions: Your ccA and ccb have 7 rows. However, it seems as your cones are of dimensions 2 and 3, which does not add up to 7. So either your cones are not the correct size, or a cone might be missing.

Cheers, Mikkel

[StoffelOdw]

Hi Mikkel,

thank you for helping me!

First: you are absolutely right - q is defined to be f and f (resp. q) should be a column vector. I changed this in my code by

q = f' # transpose row vector f to give column vector q

You are also right about the dimensions of the cones. Due to the mathematical formulation of my problem, I have to do a transformation, and obviously I made a mistake here. I still did not find my transformation error, but to get the dimensions right (just to fix the stackoverflow), I set the dimension of the nonnegative cone from 2 to 4, thus cones is now

2-element Vector{Clarabel.SupportedCone}:
 Clarabel.NonnegativeConeT(4)
 Clarabel.SecondOrderConeT(3)

I also changed ccAslightly to

7×3 SparseMatrixCSC{Float64, Int64} with 14 stored entries:
  1.0          ⋅          ⋅ 
   ⋅          1.0         ⋅ 
  0.015       0.05       1.0
 -1.0        -1.0       -1.0
  0.303653    0.443741    ⋅ 
 -0.240127    0.151639    ⋅ 
  0.0115516  -0.721179    ⋅ 

I hoped to get rid of the stackoverflow this way (although the mathematical solution to my problem might not be correct, yet). But it did not work. I still the the stackoverflow:

StackOverflowError:

Stacktrace:
     [1] Array
       @ ./boot.jl:477 [inlined]
     [2] Array
       @ ./boot.jl:486 [inlined]
     [3] zeros
       @ ./array.jl:636 [inlined]
     [4] zeros
       @ ./array.jl:632 [inlined]
     [5] Dict{Symbol, Union{}}()
       @ Base ./dict.jl:70
     [6] Dict{Symbol, Union{}}(kv::@Kwargs{})
       @ Base ./dict.jl:81
     [7] dict_with_eltype
       @ ./abstractdict.jl:580 [inlined]
     [8] Dict(kv::@Kwargs{})
       @ Base ./dict.jl:109
     [9] setup!(s::Clarabel.Solver{Float64}, P::Matrix{Float64}, c::Adjoint{Float64, Matrix{Float64}}, A::SparseMatrixCSC{Float64, Int64}, b::Vector{Float64}, cones::Vector{Clarabel.SupportedCone}; kwargs::@Kwargs{})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:71
    [10] setup!(s::Clarabel.Solver{Float64}, P::Matrix{Float64}, c::Adjoint{Float64, Matrix{Float64}}, A::SparseMatrixCSC{Float64, Int64}, b::Vector{Float64}, cones::Vector{Clarabel.SupportedCone})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:69
    [11] setup!(s::Clarabel.Solver{Float64}, P::Matrix{Float64}, c::Adjoint{Float64, Matrix{Float64}}, A::SparseMatrixCSC{Float64, Int64}, b::Vector{Float64}, cones::Vector{Clarabel.SupportedCone}; kwargs::@Kwargs{})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:72
--- the last 2 lines are repeated 39986 more times ---
 [79984] setup!(s::Clarabel.Solver{Float64}, P::Matrix{Float64}, c::Adjoint{Float64, Matrix{Float64}}, A::SparseMatrixCSC{Float64, Int64}, b::Vector{Float64}, cones::Vector{Clarabel.SupportedCone})
       @ Clarabel ~/.julia/packages/Clarabel/gIzj0/src/solver.jl:69

Maybe you have any idea??

Thank you and kind regards Stoffel

[mipals]

Hi Stoffel,

I think you're hitting one of the "quirks" of Julia for new users. Namely that transposing wraps the 1x3 matrix f by an adjoint type instead of copying the data into a Vector. What you should do instead is something akin to q = vec(f) as now q will be a Vector. Note that q and f here will refer to the same underlying data, so changing q will change f and vice versa. If you instead want to copy the data you should use q = f[:] .

[StoffelOdw]

Hi Mikkel,

THANK YOU SO MUCH!!! I am completly new to Julia, so I did not realize my mistake. Your last comment helped me very, very much!

During the problem transformation, I do a lot of matrix and vector transpose (using ' like in f'), to build block matrices. After your comment, I realized, that my resulting matrices used adjoint types. I changed that using sparse( .... )and know ... everything works fine :-)

Thank you again for your help!!!

Cheers Stoffel

[mipals]

Hi Stoffel,

Glad to hear that it worked!

Note that there is nothing inherently wrong with using the adjoint types, but sometimes they give issues with dispatching (as was the case above).

Hopefully you also manage to find the error in your transformations 😄

[StoffelOdw]

Hi Mikkel,

thanks again for your help. I will definitely dive deeper into the typesystem of Julia, so that I better understand these kind of errors in the future. Meanwhile, I was able to fix the error in my transformation. Luckily, I managed to get the same results using Clarabel and Julia, as I got before using Matlab.

Have a great weekend!

Kind regards Uwe