Add GPU tests

[github-actions[bot]]

Package name	latest	stable
CaNNOLeS.jl
DCISolver.jl
DerivativeFreeSolvers.jl
JSOSolvers.jl
NLPModelsIpopt.jl
OptimizationProblems.jl
Percival.jl
QuadraticModels.jl
SolverBenchmark.jl
SolverTools.jl

[tmigot]

Ok, so there seems to be 2 issues:

• [ ] hprod
• [ ] sparse Jacobian/Hessian

For the first one, a reproducible script:

using CUDA, ADNLPModels, NLPModels

hs6_autodiff(::Type{T}; kwargs...) where {T <: Number} = hs6_autodiff(Vector{T}; kwargs...)
function hs6_autodiff(::Type{S} = Vector{Float64}; kwargs...) where {S}
  x0 = S([-12 // 10; 1])
  f(x) = (1 - x[1])^2
  c(x) = [10 * (x[2] - x[1]^2)]
  lcon = fill!(S(undef, 1), 0)
  ucon = fill!(S(undef, 1), 0)

  return ADNLPModel(f, x0, c, lcon, ucon, name = "hs6_autodiff"; kwargs...)
end

nlp = hs6_autodiff(CuArray{Float64})
CUDA.allowscalar()
jth_hprod(nlp, nlp.meta.x0, nlp.meta.x0, 1) # same for hprod(nlp, nlp.meta.x0, nlp.meta.x0)
#=
ERROR: GPU compilation of MethodInstance for (::GPUArrays.var"#map_kernel#38"{…})(::CUDA.CuKernelContext, ::CuDeviceVector{…}, ::Base.Broadcast.Broadcasted{…}, ::Int64) failed
KernelError: passing and using non-bitstype argument

Argument 4 to your kernel function is of type Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Tuple{Base.OneTo{Int64}}, ForwardDiff.var"#85#86"{ForwardDiff.Tag{ADNLPModels.var"#lag#141"{Int64, var"#f#6", Int64, ADNLPModels.var"#c!#319"{var"#c#7"}, ADNLPModels.var"#lag#134#142"}, Float64}}, Tuple{Base.Broadcast.Extruded{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ADNLPModels.var"#lag#141"{Int64, var"#f#6", Int64, ADNLPModels.var"#c!#319"{var"#c#7"}, ADNLPModels.var"#lag#134#142"}, Float64}, Float64, 1}}, Tuple{Bool}, Tuple{Int64}}}}, which is not isbits:
  .args is of type Tuple{Base.Broadcast.Extruded{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ADNLPModels.var"#lag#141"{Int64, var"#f#6", Int64, ADNLPModels.var"#c!#319"{var"#c#7"}, ADNLPModels.var"#lag#134#142"}, Float64}, Float64, 1}}, Tuple{Bool}, Tuple{Int64}}} which is not isbits.
    .1 is of type Base.Broadcast.Extruded{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ADNLPModels.var"#lag#141"{Int64, var"#f#6", Int64, ADNLPModels.var"#c!#319"{var"#c#7"}, ADNLPModels.var"#lag#134#142"}, Float64}, Float64, 1}}, Tuple{Bool}, Tuple{Int64}} which is not isbits.
      .x is of type Vector{ForwardDiff.Dual{ForwardDiff.Tag{ADNLPModels.var"#lag#141"{Int64, var"#f#6", Int64, ADNLPModels.var"#c!#319"{var"#c#7"}, ADNLPModels.var"#lag#134#142"}, Float64}, Float64, 1}} which is not isbits.
=#

For the second one, this seems to be an issue with Symbolics:

using CUDA, ADNLPModels, NLPModels, Symbolics

hs6_autodiff(::Type{T}; kwargs...) where {T <: Number} = hs6_autodiff(Vector{T}; kwargs...)
function hs6_autodiff(::Type{S} = Vector{Float64}; kwargs...) where {S}
  x0 = S([-12 // 10; 1])
  f(x) = (1 - x[1])^2
  c(x) = [10 * (x[2] - x[1]^2)]
  lcon = fill!(S(undef, 1), 0)
  ucon = fill!(S(undef, 1), 0)

  return ADNLPModel(f, x0, c, lcon, ucon, name = "hs6_autodiff"; kwargs...)
end
S = CuArray{Float64}
function c!(cx, x)
    cx .= [10 * (x[2] - x[1]^2)]
    return cx
end
x0 = S([-12 // 10; 1])
output = similar(x0, 1)
# nlp = hs6_autodiff(CuArray{Float64})
# ADNLPModels.SparseADJacobian(2, x -> (1 - x[1])^2, 1, c!, x0 = x0)
# J = ADNLPModels.compute_jacobian_sparsity(c!, output, x0)
J = Symbolics.jacobian_sparsity(c!, cx, x0)

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.99%. Comparing base (30cfab4) to head (d53a266). Report is 1 commits behind head on main.

:exclamation: Current head d53a266 differs from pull request most recent head d0562a9. Consider uploading reports for the commit d0562a9 to get more accurate results

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #222 +/- ## ========================================== - Coverage 95.21% 94.99% -0.22% ========================================== Files 14 14 Lines 1650 1558 -92 ========================================== - Hits 1571 1480 -91 + Misses 79 78 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[amontoison]

@tmigot We should install buildkite to easily test on GPU. I will request it today.

[tmigot]

Thanks @amontoison for the update and the automated CI ! That's awesome. Since, we won't solve the two problems right now, I suggest we merge this and create issues.

[amontoison]

Thanks @amontoison for the update and the automated CI ! That's awesome. Since, we won't solve the two problems right now, I suggest we merge this and create issues.

@tmigot Can you just fix the following error: https://buildkite.com/julialang/adnlpmodels-dot-jl/builds/3#018f3fd0-cb5c-418d-bee8-059d789fb60f/107-473 ??

We merge the PR after that.

[tmigot]

Oh, I missed that you moved the GPU tests in a different file. I will check that