PeriodicKernel fails due to 'Mutating arrays is not supported'

[sdwfrost]

If I change the kernel in Regression - Gaussian.ipynb to a PeriodicKernel(), I get the following error:

Mutating arrays is not supported

Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] (::Zygote.var"#455#456")(::Nothing) at /home/simon/.julia/packages/Zygote/1GXzF/src/lib/array.jl:64
 [3] (::Zygote.var"#999#back#457"{Zygote.var"#455#456"})(::Nothing) at /home/simon/.julia/packages/ZygoteRules/6nssF/src/adjoint.jl:49
 [4] _pairwise! at /home/simon/.julia/packages/Distances/mNZEh/src/generic.jl:117 [inlined]
 [5] (::typeof(∂(_pairwise!)))(::Array{Float64,2}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [6] #pairwise!#3 at /home/simon/.julia/packages/Distances/mNZEh/src/generic.jl:182 [inlined]
 [7] (::typeof(∂(#pairwise!#3)))(::Array{Float64,2}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0 (repeats 2 times)
 [8] #pairwise#5 at /home/simon/.julia/packages/Distances/mNZEh/src/generic.jl:213 [inlined]
 [9] (::typeof(∂(#pairwise#5)))(::Array{Float64,2}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0 (repeats 2 times)
 [10] pairwise at /home/simon/.julia/packages/KernelFunctions/OAopf/src/utils.jl:74 [inlined]
 [11] (::typeof(∂(pairwise)))(::Array{Float64,2}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [12] kernelmatrix at /home/simon/.julia/packages/KernelFunctions/OAopf/src/matrix/kernelmatrix.jl:88 [inlined]
 [13] (::typeof(∂(kernelmatrix)))(::LinearAlgebra.Diagonal{Float64,Array{Float64,1}}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [14] #kernelmatrix#50 at /home/simon/.julia/packages/KernelFunctions/OAopf/src/matrix/kernelmatrix.jl:126 [inlined]
 [15] (::typeof(∂(#kernelmatrix#50)))(::LinearAlgebra.Diagonal{Float64,Array{Float64,1}}) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0 (repeats 2 times)
 [16] _∇L_ρ_reverse at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/hyperparameter/zygote_rules.jl:22 [inlined]
 [17] (::typeof(∂(_∇L_ρ_reverse)))(::Float64) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [18] #172 at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/hyperparameter/zygote_rules.jl:13 [inlined]
 [19] (::typeof(∂(λ)))(::Float64) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
 [20] (::Zygote.var"#50#51"{Zygote.Params,Zygote.Context,typeof(∂(λ))})(::Float64) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:177
 [21] gradient(::Function, ::Zygote.Params) at /home/simon/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:54
 [22] ∇L_ρ_reverse(::Function, ::AugmentedGaussianProcesses._SVGP{Float64}, ::SubArray{Float64,2,Array{Float64,2},Tuple{Array{Int64,1},Base.Slice{Base.OneTo{Int64}}},false}, ::Array{Float64,1}, ::Array{Float64,1}, ::AnalyticVI{Float64,1}, ::AugmentedGaussianProcesses.AVIOptimizer{Float64,Descent}) at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/hyperparameter/zygote_rules.jl:12
 [23] update_hyperparameters!(::AugmentedGaussianProcesses._SVGP{Float64}, ::SubArray{Float64,2,Array{Float64,2},Tuple{Array{Int64,1},Base.Slice{Base.OneTo{Int64}}},false}, ::Array{Float64,1}, ::Array{Float64,1}, ::AnalyticVI{Float64,1}, ::AugmentedGaussianProcesses.AVIOptimizer{Float64,Descent}) at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/hyperparameter/autotuning.jl:60
 [24] _broadcast_getindex_evalf at ./broadcast.jl:631 [inlined]
 [25] _broadcast_getindex at ./broadcast.jl:604 [inlined]
 [26] getindex at ./broadcast.jl:564 [inlined]
 [27] copy at ./broadcast.jl:854 [inlined]
 [28] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1},Nothing,typeof(AugmentedGaussianProcesses.update_hyperparameters!),Tuple{Tuple{AugmentedGaussianProcesses._SVGP{Float64}},Array{SubArray{Float64,2,Array{Float64,2},Tuple{Array{Int64,1},Base.Slice{Base.OneTo{Int64}}},false},1},Tuple{Array{Float64,1}},Tuple{Array{Float64,1}},Array{Any,1},Tuple{AugmentedGaussianProcesses.AVIOptimizer{Float64,Descent}}}}) at ./broadcast.jl:820
 [29] macro expansion at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/hyperparameter/autotuning.jl:15 [inlined]
 [30] update_hyperparameters!(::Type{SimpleTraits.Not{AugmentedGaussianProcesses.IsFull{SVGP{Float64,GaussianLikelihood{Float64,Nothing,Array{Float64,1}},AnalyticVI{Float64,1},1}}}}, ::SVGP{Float64,GaussianLikelihood{Float64,Nothing,Array{Float64,1}},AnalyticVI{Float64,1},1}) at /home/simon/.julia/packages/SimpleTraits/pDUSq/src/SimpleTraits.jl:331
 [31] update_hyperparameters! at /home/simon/.julia/packages/SimpleTraits/pDUSq/src/SimpleTraits.jl:338 [inlined]
 [32] train!(::SVGP{Float64,GaussianLikelihood{Float64,Nothing,Array{Float64,1}},AnalyticVI{Float64,1},1}, ::Int64; callback::Nothing, convergence::Nothing) at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/training.jl:41
 [33] train!(::SVGP{Float64,GaussianLikelihood{Float64,Nothing,Array{Float64,1}},AnalyticVI{Float64,1},1}, ::Int64) at /home/simon/.julia/packages/AugmentedGaussianProcesses/wxtsL/src/training.jl:19
 [34] macro expansion at ./util.jl:175 [inlined]

[theogf]

Hi! Thanks for trying! This a known bug due to KernelFunctions.jl : https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/issues/116

I will try to make a fix once I am back from vacations!

[sdwfrost]

I noticed that there was some activity on KernelFunctions with periodic_transform recently, which reminded me to check back to see if there was anything I can do to help fix.

[theogf]

The periodic_transform is a different thing unfortunately. However now I managed to implement kernel parameters optimization with ForwardDiff, just set AugmentedGaussianProcesses.setadbackend(:forward_diff) before running your algorithm. It's kinda of a new thing but it should work! Since ForwardDiff supports mutation it should work with PeriodicKernel