Investigate test failures

[codecov[bot]]

Codecov Report

Merging #441 (8acff99) into master (ddca4f8) will decrease coverage by 6.22%. Report is 29 commits behind head on master. The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #441      +/-   ##
==========================================
- Coverage   79.89%   73.68%   -6.22%     
==========================================
  Files          16       22       +6     
  Lines        2567     2930     +363     
==========================================
+ Hits         2051     2159     +108     
- Misses        516      771     +255     

Files Changed	Coverage Δ
lib/SurrogatesMOE/src/SurrogatesMOE.jl	0.00% <0.00%> (ø)
src/Radials.jl	84.48% <ø> (+1.72%)	:arrow_up:

... and 13 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[sathvikbhagavan]

@ChrisRackauckas

I investigated a bit about the Surrogates test failures and wanted to summarize them here:

1. Tests in SurrogatesMOE fail spuriously - i. Successful run - https://github.com/SciML/Surrogates.jl/actions/runs/6248364164?pr=441 ii. Unsuccessful run - https://github.com/SciML/Surrogates.jl/actions/runs/6248569007?pr=441 This happens when one of the clusters do not have any test points which is not handled (not 100% sure)

2. using QuasiMonteCarlo 0.2.19 (latest of 0.2) fails tests in GEKPLS tests. But using 0.2.16 works and passes all tests. The reason explained in https://github.com/SciML/Surrogates.jl/pull/420

3. Cannot bump QuasiMonteCarlo to 0.3 as for some reason https://github.com/SciML/QuasiMonteCarlo.jl/blob/master/src/Section.jl is commented and it complains about SectionSample not defined.

4. I had to bump the compat of ExtendableSparse to 1 as I get this error ERROR: MethodError: \(::Symmetric{Float64, ExtendableSparseMatrix{Float64, Int64}}, ::Vector{Float64}) is ambiguous with 0.6 in julia 1.9. After bumping, it works.

So, I think (1) can be fixed but I am not sure what's happening with QuasiMonteCarlo currently.

[ChrisRackauckas]

Okay, we should just bound QMC to 0.2.16, set the RNG seed for 1. Let's see how far that takes us.

[sathvikbhagavan]

I think the reason MOE tests were failing spuriously as division between train and test data was not same each run because of using BitArray(undef, n) in https://github.com/SciML/Surrogates.jl/blob/master/lib/SurrogatesMOE/src/SurrogatesMOE.jl#L144 as all elements are not false all the time (we want all of them to be false)

This is fixed in 6848450

julia> for _ in 1:10
              @info sum(BitArray(undef, 150))
              end
[ Info: 26
[ Info: 49
[ Info: 4
[ Info: 3
[ Info: 25
[ Info: 49
[ Info: 7
[ Info: 3
[ Info: 51
[ Info: 48