Closed LorenzoBottaccioli closed 7 months ago
It should be
@named ode_system = NeuralPDE.ODESystem(eqs, t, [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs])
[t]
-> t
@sathvikbhagavan That solved only that line of code, the script still not working.
MethodError: no method matching discretize(::ODESystem, ::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, typeof(NeuralPDE.numeric_derivative), Bool, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})
Closest candidates are:
discretize(::PDESystem, ::PhysicsInformedNN)
@ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:682
Stacktrace:
[1] top-level scope
@ In[9]:1
Yes, PhysicsInformedNN
only works with PDESystem
and not ODESystem
.
I am not sure what exactly are you trying to solve here. Can you elaborate?
One way might be to use NNODE
by converting ODESystem
into ODEProblem
@sathvikbhagavan I kneed to discover unknown parameter of an ODE, I was thinking to use the param_estim=true
of NeuralPDE.PhysicsInformedNN
.
The equation is
[Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]
whereR,C, Cs
are unknown, al the other are time-dependent variables that are measured every 10 minutes.
@LorenzoBottaccioli you could set the ODE eqs as a PDEsystem(0 instead of an ODESystem() in your code and have an appropriate additional loss function for parameter estimation in this case, refer Docs.
@LorenzoBottaccioli, is it necessary for you to use a PINN? Looking at your problem statement, it seems to be a simple optimization problem for finding parameters. Is there a reason you want to train a PINN as well?
@sathvikbhagavan I need an NN to make predictions, this is way I wanted to use a PINN. But I don't have the parameters of the ODE so I need to find them and I was trying to do the things together. @AstitvaAggarwal I have tried to do what you have suggested already but I still get an error, I have tried to do what is written in the docs but it doesn't work. Here is the code:
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL
# Definisci il modello dell'equazione differenziale
@variables t::Real
@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R::Real C::Real Cs::Real
Dt = Differential(t)
eqs = [Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]
tspan = (0.0, 365.0 * 24.0 * 60.0) # Dati per un anno
data = CSV.File("shoebox_free.csv") |> DataFrame
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"
t_data = 1:size(data, 1)
dataloader = DataLoader([vcat(T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])
input_dim = length(tspan)
hidden_units = 8
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, 1))
dt = 600.0 # 600 secondi (10 minuti)
discretization = NeuralPDE.PhysicsInformedNN([chain1],
NeuralPDE.GridTraining(dt),
param_estim = true)
domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~19.3]
@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))
prob = NeuralPDE.discretize(pde_system, discretization)
function additional_loss(phi, θ, p)
T_in_predict = phi[1](t_data, θ[1])[1]
return sum(abs2(T_in_predict .- T_in_data) / length(data))
end
res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))
p_optimized = res.u[end]
minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)
using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)
Here the error:
type NamedTuple has no field T_ext
Stacktrace:
[1] getproperty
@ .\Base.jl:37 [inlined]
[2] getindex(#unused#::ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}, s::Symbol)
@ ComponentArrays C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\axis.jl:157
[3] _broadcast_getindex_evalf
@ .\broadcast.jl:683 [inlined]
[4] _broadcast_getindex
@ .\broadcast.jl:656 [inlined]
[5] (::Base.Broadcast.var"#31#32"{Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(getindex), Tuple{Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}, Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(ComponentArrays.getval), Tuple{Tuple{DataType}}}}}})(k::Int64)
@ Base.Broadcast .\broadcast.jl:1088
[6] ntuple
@ .\ntuple.jl:48 [inlined]
[7] copy
@ .\broadcast.jl:1088 [inlined]
[8] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(getindex), Tuple{Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}, Base.Broadcast.Broadcasted{Base.Broadcast.Style{Tuple}, Nothing, typeof(ComponentArrays.getval), Tuple{Tuple{DataType}}}}})
@ Base.Broadcast .\broadcast.jl:873
[9] #s20#55
@ C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\array_interface.jl:122 [inlined]
[10] var"#s20#55"(::Any, index_fun::Any, x::Any, idx::Any)
@ ComponentArrays .\none:0
[11] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any})
@ Core .\boot.jl:602
[12] getproperty
@ C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\namedtuple_interface.jl:14 [inlined]
[13] rrule(#unused#::typeof(getproperty), x::ComponentArrays.ComponentVector{Float64, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}}, s::Symbol)
@ ComponentArrays C:\Users\loren\.julia\packages\ComponentArrays\Cy8Rn\src\compat\chainrulescore.jl:8
[14] rrule
@ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
[15] chain_rrule
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
[16] macro expansion
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[17] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[18] macro expansion
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\literal_getproperty.jl:93 [inlined]
[19] _pullback(ctx::Zygote.Context{false}, #unused#::typeof(ZygoteRules.literal_getproperty), x::ComponentArrays.ComponentVector{Float64, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{ComponentArrays.Axis{(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)}}}, f::Val{:T_ext})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\literal_getproperty.jl:117
[20] macro expansion
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
[21] macro expansion
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
[22] macro expansion
@ .\none:0 [inlined]
[23] _pullback
@ .\none:0 [inlined]
[24] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[25] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[26] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[27] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[28] _pullback
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
[29] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[30] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
[31] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[32] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
[33] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[34] _pullback
@ .\none:0 [inlined]
[35] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[36] #661
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
[37] iterate
@ .\generator.jl:47 [inlined]
[38] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
@ Base .\array.jl:802
[39] collect_similar
@ .\array.jl:711 [inlined]
[40] map
@ .\abstractarray.jl:3261 [inlined]
[41] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
[42] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
[43] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
[44] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[45] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[46] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[47] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[48] _pullback
@ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
[49] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[50] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[51] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[52] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[53] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
[54] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[55] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[56] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[57] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[58] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
[59] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[60] pullback(f::Function, cx::Zygote.Context{false}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
[61] pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
[62] gradient(f::Function, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
[63] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}})(::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
[64] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}})(G::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, θ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
[65] value_gradient!!(obj::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
[66] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}}, d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
[67] optimize(d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}, ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"}}})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
[68] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
[69] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#9#10"})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
[70] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x295b6845, 0xce1110b6, 0xa17beb73, 0xc449b733, 0xb3d6896e), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xb5fb80bb, 0x61a62589, 0x2ec55669, 0x07bea97e, 0xb7ae3a52), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, Vector{NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}}, Nothing, Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:177, Axis(T_in = ViewAxis(1:177, Axis(layer_1 = ViewAxis(1:24, Axis(weight = ViewAxis(1:16, ShapedAxis((8, 2), NamedTuple())), bias = ViewAxis(17:24, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(25:96, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(97:168, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(169:177, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))),)), p = 178:180)}}}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:maxiters, :callback), Tuple{Int64, var"#9#10"}}})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:8
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL
# Definisci il modello dell'equazione differenziale
@variables t::Real
@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R::Real C::Real Cs::Real
#details of problem to be solved
Dt = Differential(t)
eqs = [Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]
tspan = (0.0, 365.0 * 24.0 * 60.0) # Dati per un anno
domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~19.3]
# dataset creation and additional loss function
data = CSV.File("shoebox_free.csv") |> DataFrame
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"
t_data = 1:size(data, 1)
dataloader = DataLoader([vcat(T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])
function additional_loss(phi, θ, p)
T_in_predict = phi[1](t_data, θ[1])[1]
return sum(abs2(T_in_predict .- T_in_data) / length(data))
end
# Creating chain
input_dim = length(tspan)
hidden_units = 8
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, 1))
# discretize domain with PINN
dt = 600.0 # 600 secondi (10 minuti)
discretization = NeuralPDE.PhysicsInformedNN([chain1],
NeuralPDE.GridTraining(dt),
param_estim = true, additional_loss = additional_loss)
@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))
prob = NeuralPDE.discretize(pde_system, discretization)
# solve
res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))
# checking solution
p_optimized = res.u[end]
minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)
using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)
@LorenzoBottaccioli try using this(ive just passed the additional loss funciton in PhysicsInformedNN() and formatted the code), the error is mostly due to some wrong argument being passed
@AstitvaAggarwal thanks for your help. I dont get how I could solve the error of type NamedTuple has no filed T_ext
. Here if you want there is the input data
shoebox_free.csv
@LorenzoBottaccioli You are mostly defining the additional loss function incorrectly. (something like below from tutorial)
depvars = [:x, :y, :z]
function additional_loss(phi, θ, p)
return sum(sum(abs2, phi[i](t_, θ[depvars[i]]) .- u_[[i], :]) / len for i in 1:1:3)
end
also Q_heating_data etc after T_ext_data are all zero datavectors.
@AstitvaAggarwal still I'm not understanding the tutorial at least for me it is not very clear and explanatory. The other data are zeros I know. The case study is shoebox building that is running in free-float, hence no heating or cooling so far and no people inside, just solar radiation as gain. What I'm not getting in general is how I pass the data from T_et of any other feature to the PINN problem.
@AstitvaAggarwal @sathvikbhagavan I have re-arranged the code in a more comprehensive way but I get a new error. Can you help me in solving it?
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets, Optimization, OptimizationOptimJL, DataInterpolations
# Definisci il modello dell'equazione differenziale
#@variables t::Real
@syms T_in(t)::Real
@syms T_ext(t)::Real
@syms Q_heating(t)::Real
@syms Q_cooling(t)::Real
@syms Q_sun(t)::Real
@syms Q_lights(t)::Real
@syms Q_equipment(t)::Real
@parameters R, C, Cs, t
Dt = Differential(t)
eqs = [Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C + Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C]
domains = [t ∈ (0.0,365.0 * 24.0 * 60.0)]
bcs=[Dt(T_in(0))~4.48]
dt = 600.0 # 600 seconds (10 minute)
# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0) # Dati per un anno
# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame
# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"
t_data = 1:size(data, 1)
dataloader = DataLoader([vcat(T_in_data,T_ext_data, Q_heating_data, Q_cooling_data, Q_sun_data, Q_lights_data, Q_equipment_data, t_data)])
# Define the NN
input_dim = 7
hidden_units = 8
len=length(t_data)
chain1 = Lux.Chain(Lux.Dense(input_dim, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, hidden_units, Lux.σ),
Lux.Dense(hidden_units, 1))
#Define dependent and independent vatiables
indvars=[t,T_ext,Q_heating,Q_cooling,Q_sun,Q_lights,Q_equipment]
depvars = [T_in]
# Define the loss
function additional_loss(phi, θ, p)
T_in_predict = phi(t_data, θ[depvars])
return sum(abs2(T_in_predict .- T_in_data))# / len)
end
@named pde_system=NeuralPDE.PDESystem(eqs,bcs,domains,indvars,depvars, [R, C, Cs], defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])
discretization = NeuralPDE.PhysicsInformedNN(chain1,
NeuralPDE.GridTraining(dt),
param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)
# Parameter Optimization
res = Optimization.solve(prob, BFGS(), maxiters = 5000, callback = (p, l) -> println("Current loss is: $l"))
p_optimized = res.u[end]
# Plot fo results
minimizer = res.u.depvar[1]
T_in_predict = minimizer(t_data)
using Plots
plot(t_data, T_in_data, label = "Dati Osservati")
plot!(t_data, T_in_predict, label = "Temperatura Prevista", linestyle = :dash)
I get the following error. What I still don't understand is when and how I should use the data I have loaded.
BoundsError: attempt to access 1×877 Matrix{Float64} at index [[2], 1:877]
Stacktrace:
[1] throw_boundserror(A::Matrix{Float64}, I::Tuple{Vector{Int64}, Base.Slice{Base.OneTo{Int64}}})
@ Base .\abstractarray.jl:744
[2] checkbounds
@ .\abstractarray.jl:709 [inlined]
[3] _getindex
@ .\multidimensional.jl:860 [inlined]
[4] getindex
@ .\abstractarray.jl:1294 [inlined]
[5] rrule(::typeof(getindex), ::Matrix{Float64}, ::Vector{Int64}, ::Function)
@ ChainRules C:\Users\loren\.julia\packages\ChainRules\snrkz\src\rulesets\Base\indexing.jl:63
[6] rrule
@ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
[7] chain_rrule
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
[8] macro expansion
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[9] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[10] macro expansion
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
[11] macro expansion
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
[12] macro expansion
@ .\none:0 [inlined]
[13] _pullback
@ .\none:0 [inlined]
[14] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[15] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[16] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[17] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[18] _pullback
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
[19] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[20] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
[21] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, ::Matrix{Float64}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[22] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
[23] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[24] _pullback
@ .\none:0 [inlined]
[25] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[26] #661
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
[27] iterate
@ .\generator.jl:47 [inlined]
[28] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
@ Base .\array.jl:802
[29] collect_similar
@ .\array.jl:711 [inlined]
[30] map
@ .\abstractarray.jl:3261 [inlined]
[31] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
[32] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
[33] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
[34] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[35] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[36] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[37] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[38] _pullback
@ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
[39] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[40] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[41] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[42] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[43] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
[44] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[45] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[46] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[47] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[48] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
[49] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[50] pullback(f::Function, cx::Zygote.Context{false}, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
[51] pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
[52] gradient(f::Function, args::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
[53] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}})(::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
[54] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}})(G::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, θ::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
[55] value_gradient!!(obj::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
[56] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}}, d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
[57] optimize(d::TwiceDifferentiable{Float64, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentMatrix{Float64, Matrix{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}, ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}}, initial_x::ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"}}})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
[58] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
[59] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, var"#3#4"})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
[60] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0xad10bb64, 0x1d2657c2, 0xc9c67c1a, 0xbaf4020a, 0xc4a1861b), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x7cf59a19, 0x0162d753, 0x260a99e5, 0xc4bc113e, 0x070b6155), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Lux.Chain{NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), Tuple{Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(sigmoid_fast), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}, Lux.Dense{true, typeof(identity), typeof(Lux.glorot_uniform), typeof(Lux.zeros32)}}}}, NamedTuple{(:layer_1, :layer_2, :layer_3, :layer_4), NTuple{4, NamedTuple{(), Tuple{}}}}}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ComponentArrays.ComponentVector{Float64, Vector{Float64}, Tuple{ComponentArrays.Axis{(depvar = ViewAxis(1:217, Axis(layer_1 = ViewAxis(1:64, Axis(weight = ViewAxis(1:56, ShapedAxis((8, 7), NamedTuple())), bias = ViewAxis(57:64, ShapedAxis((8, 1), NamedTuple())))), layer_2 = ViewAxis(65:136, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_3 = ViewAxis(137:208, Axis(weight = ViewAxis(1:64, ShapedAxis((8, 8), NamedTuple())), bias = ViewAxis(65:72, ShapedAxis((8, 1), NamedTuple())))), layer_4 = ViewAxis(209:217, Axis(weight = ViewAxis(1:8, ShapedAxis((1, 8), NamedTuple())), bias = ViewAxis(9:9, ShapedAxis((1, 1), NamedTuple())))))), p = 218:220)}}}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:maxiters, :callback), Tuple{Int64, var"#3#4"}}})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:81
hey, sorry got busy with some other work. taking a look at this now. also while your at it why not join our slack? link , make sure to join the sciml-bridged channel.
hey @LorenzoBottaccioli im confused about your problem formulation, you have mentioned indvars=[t,T_ext,Q_heating,Q_cooling,Q_sun,Q_lights,Q_equipment] depvars = [T_in] but in your equations every variable seems to be a function of t only, so wouldnt you make only t your independant variable?
@LorenzoBottaccioli this works for me, ive also reformulated the code as mentioned above.
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
Optimization, OptimizationOptimJL
# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs
@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)
# R, C, Cs = [1, 2, 3]
Dt = Differential(t)
# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_ext(t) - T_in(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C
domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]
dt = 600.0 # 600 seconds (10 minute)
# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0) # Dati per un anno
# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame
# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]
t_data = collect(Float64, 1:size(data, 1))
dataloader = DataLoader([
vcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data,
t_data),
])
# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)
chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain2 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain3 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain4 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain5 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain6 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
chain7 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
#Define dependent and independent vatiables
indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data)
# Define the loss(additional loss will be using all data vectors)
init_params = [Flux.destructure(c)[1]
for c in [chain1, chain2, chain3, chain4, chain5, chain6, chain7]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]
function additional_loss(phi, θ, p)
return sum(sum(abs2, phi[i](t_data[1:500]', θ[sep[i]]) .- u_[:, [i]]') / 500.0
for i in 1:1:7)
end
@named pde_system = NeuralPDE.PDESystem(eqs,
bcs,
domains,
[t],
[T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)
],
[R, C, Cs],
defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])
discretization = NeuralPDE.PhysicsInformedNN([
chain1,
chain2,
chain3,
chain4,
chain5,
chain6, chain7,
],
NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)
# Parameter Optimization
res = Optimization.solve(prob,
BFGS(),
maxiters = 1000,
callback = callback)
@AstitvaAggarwal tnx. There are few things I don't get:
1) Why there is the need of 7 chains? I need only an NN to predict the T_in variation not all the others T_ext(t),Q_heating(t),Q_cooling(t),Q_sun(t),Q_lights(t),Q_equipment(t)
are exogenpous inputs that only vary with time.
2) What you have done in the additional loss?
3) When do I use the data in dataloader?
1> using chains you could train it not only at the timepoints in dataset but also on other timepoints in the domain of t. also you were taking the rest at dependant variables on t so learning that variable made sense to me. 2> additonal loss adds MSE loss for all 7 variables for its data and that particular NNs output. 3>No idea, i thought you were using that later on in your code so ignored that
@AstitvaAggarwal I tried to explain the objective better. The goal is to predict T_in
that varieties over time with the provided equation eqs
. T_in
is influenced by exogenous inputs that vary in time with some of the exogenous inputs that are controllable. So there is no need to predict any of those variables. The overall goal hence is to learn the unknown parameters of the eqs
and use PINN to predict the variation of T_in
in time influenced by the exogenous inputs.
I have tried to review the code you provided to work as I mentioned above but I get an error related to the dimension that is due to the presence of the unknown parameter.
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
Optimization, OptimizationOptimJL
# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs
@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)
# R, C, Cs = [1, 2, 3]
Dt = Differential(t)
# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_in(t)-T_ext(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C
domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]
dt = 600.0 # 600 seconds (10 minute)
# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)
# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame
# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]
t_data = collect(Float64, 1:size(data, 1))
dataloader = DataLoader([
vcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data,
t_data),
])
# Define the NN
input_dim = 7
hidden_units = 8
len = length(t_data)
chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),Flux.Dense(hidden_units, hidden_units, σ), Flux.Dense(hidden_units, 1)) |>
Flux.f64
indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data)
init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]
function additional_loss(phi, θ, p)
return sum(sum(abs2, phi(t_data, θ[sep[1]]) .- u_[:,1]))
end
discretization = NeuralPDE.PhysicsInformedNN(chain1,
NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)
# Parameter Optimization
res = Optimization.solve(prob,
BFGS(),
maxiters = 1000)
Then I got
DimensionMismatch: Rebuild expected a vector of length 145, got 147
Stacktrace:
[1] #_rebuild#48
@ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:95 [inlined]
[2] _rebuild
@ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:94 [inlined]
[3] #rrule#53
@ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:136 [inlined]
[4] rrule
@ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:134 [inlined]
[5] rrule
@ C:\Users\loren\.julia\packages\ChainRulesCore\7MWx2\src\rules.jl:134 [inlined]
[6] chain_rrule
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\chainrules.jl:223 [inlined]
[7] macro expansion
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[8] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:101 [inlined]
[9] _pullback
@ C:\Users\loren\.julia\packages\Optimisers\er7zv\src\destructure.jl:59 [inlined]
[10] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:371 [inlined]
[11] _pullback(::Zygote.Context{false}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::Matrix{Float64}, ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[12] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:375 [inlined]
[13] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#12#13", ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[14] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\pinn_types.jl:410 [inlined]
[15] _pullback(::Zygote.Context{false}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::NeuralPDE.var"#12#13", ::Matrix{Float64}, ::Vector{Vector{Float64}}, ::Int64, ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[16] macro expansion
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:169 [inlined]
[17] macro expansion
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:163 [inlined]
[18] macro expansion
@ .\none:0 [inlined]
[19] _pullback
@ .\none:0 [inlined]
[20] _pullback(::Zygote.Context{false}, ::typeof(RuntimeGeneratedFunctions.generated_callfunc), ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[21] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[22] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[23] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[24] _pullback
@ C:\Users\loren\.julia\packages\RuntimeGeneratedFunctions\Yo8zx\src\RuntimeGeneratedFunctions.jl:150 [inlined]
[25] _pullback(::Zygote.Context{false}, ::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, ::Matrix{Float64}, ::Vector{Float64}, ::NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, ::typeof(NeuralPDE.numeric_derivative), ::NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, ::NeuralPDE.var"#12#13", ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[26] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:192 [inlined]
[27] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, ::Matrix{Float64}, ::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[28] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\training_strategies.jl:48 [inlined]
[29] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}, args::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[30] _pullback
@ .\none:0 [inlined]
[31] _pullback(ctx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{Vector{Float64}}, args::NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[32] #661
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183 [inlined]
[33] iterate
@ .\generator.jl:47 [inlined]
[34] _collect(c::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, itr::Base.Generator{Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Zygote.var"#661#665"{Zygote.Context{false}, NeuralPDE.var"#294#309"{Vector{Float64}}}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
@ Base .\array.jl:802
[35] collect_similar
@ .\array.jl:711 [inlined]
[36] map
@ .\abstractarray.jl:3261 [inlined]
[37] ∇map(cx::Zygote.Context{false}, f::NeuralPDE.var"#294#309"{Vector{Float64}}, args::Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:183
[38] _pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\array.jl:226 [inlined]
[39] _pullback
@ C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:592 [inlined]
[40] _pullback(::Zygote.Context{false}, ::NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, ::Vector{Float64}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[41] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[42] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[43] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[44] _pullback
@ C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\scimlfunctions.jl:3580 [inlined]
[45] _pullback(::Zygote.Context{false}, ::OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ::Vector{Float64}, ::SciMLBase.NullParameters)
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[46] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[47] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[48] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[49] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:72 [inlined]
[50] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}, args::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[51] _apply(::Function, ::Vararg{Any})
@ Core .\boot.jl:838
[52] adjoint
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\lib\lib.jl:203 [inlined]
[53] _pullback
@ C:\Users\loren\.julia\packages\ZygoteRules\4nXuu\src\adjoint.jl:66 [inlined]
[54] _pullback
@ C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:76 [inlined]
[55] _pullback(ctx::Zygote.Context{false}, f::Optimization.var"#229#238"{Tuple{}, Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, args::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface2.jl:0
[56] pullback(f::Function, cx::Zygote.Context{false}, args::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:44
[57] pullback
@ C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:42 [inlined]
[58] gradient(f::Function, args::Vector{Float64})
@ Zygote C:\Users\loren\.julia\packages\Zygote\YYT6v\src\compiler\interface.jl:96
[59] (::Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}})(::Vector{Float64}, ::Vector{Float64})
@ Optimization C:\Users\loren\.julia\packages\Optimization\vFala\src\function\zygote.jl:74
[60] (::OptimizationOptimJL.var"#10#16"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}, OptimizationOptimJL.var"#9#15"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}})(G::Vector{Float64}, θ::Vector{Float64})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:200
[61] value_gradient!!(obj::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, x::Vector{Float64})
@ NLSolversBase C:\Users\loren\.julia\packages\NLSolversBase\kavn7\src\interface.jl:82
[62] initial_state(method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}}, d::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, initial_x::Vector{Float64})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\solvers\first_order\bfgs.jl:94
[63] optimize(d::TwiceDifferentiable{Float64, Vector{Float64}, Matrix{Float64}, Vector{Float64}}, initial_x::Vector{Float64}, method::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, options::Optim.Options{Float64, OptimizationOptimJL.var"#_cb#14"{OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"}}})
@ Optim C:\Users\loren\.julia\packages\Optim\V8ZEC\src\multivariate\optimize\optimize.jl:36
[64] __solve(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"})
@ OptimizationOptimJL C:\Users\loren\.julia\packages\OptimizationOptimJL\uRfW9\src\OptimizationOptimJL.jl:249
[65] solve!(cache::OptimizationOptimJL.OptimJLOptimizationCache{OptimizationFunction{false, Optimization.AutoZygote, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.var"#228#237"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#231#240"{Optimization.var"#227#236"{OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}}}, Optimization.var"#235#244", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Optimization.ReInitCache{Vector{Float64}, SciMLBase.NullParameters}, Nothing, Nothing, Nothing, Nothing, Nothing, BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}, Base.Iterators.Cycle{Tuple{Optimization.NullData}}, Bool, OptimizationOptimJL.var"#6#8"})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:160
[66] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, NeuralPDE.var"#full_loss_function#308"{NeuralPDE.var"#null_nonadaptive_loss#115", Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x03d798f9, 0x34fa0c63, 0xe7b60143, 0x8b2612da, 0x6b36183f), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, Vector{NeuralPDE.var"#74#75"{NeuralPDE.var"#209#210"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:cord, Symbol("##θ#293"), :phi, :derivative, :integral, :u, :p), NeuralPDE.var"#_RGF_ModTag", NeuralPDE.var"#_RGF_ModTag", (0x64128365, 0xe99c13c8, 0x3553718d, 0xc14f6af0, 0x258a1b45), Expr}, NeuralPDE.var"#12#13", NeuralPDE.var"#269#276"{NeuralPDE.var"#269#270#277"{typeof(NeuralPDE.numeric_derivative)}, Dict{Symbol, Int64}, Dict{Symbol, Int64}, GridTraining{Float64}}, typeof(NeuralPDE.numeric_derivative), NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, Vector{Float64}}, Matrix{Float64}}}, NeuralPDE.PINNRepresentation, Bool, Vector{Int64}, Int64, NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}, typeof(additional_loss), Bool, Vector{Float64}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:maxiters,), Tuple{Int64}}})
@ SciMLBase C:\Users\loren\.julia\packages\SciMLBase\QqtZA\src\solve.jl:81
okay, did some minor changes to your new code and it works but ive made T_in chain input as 1 dim as that only depends on t (time) in your variable definition also in the discretization function used [ ] for parameter estimation to work properly. also why is dt=600? your domain is in minutes anyways, also dataset points is for every 10 minutes , so you would want to break that domain down into a grid spacing of 10 mins and not 600 as 600 would imply 600 minutes here. also the code :
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
Optimization, OptimizationOptimJL
# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs
@variables T_in(..),
T_ext(..),
Q_heating(..),
Q_cooling(..),
Q_sun(..),
Q_lights(..),
Q_equipment(..)
# R, C, Cs = [1, 2, 3]
Dt = Differential(t)
# eqs = Dt(T_in(t)) ~ (-T_ext(t) + T_in(t)) / (R * C)
eqs = Dt(T_in(t)) ~ (T_in(t) - T_ext(t)) / (R * C) + Q_heating(t) / C - Q_cooling(t) / C +
Q_sun(t) / Cs + (Q_lights(t) + Q_equipment(t)) / C
domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
bcs = [Dt(T_in(0.0)) ~ 4.48]
dt = 600.0 # 600 seconds (10 minute)
# Define the temporal space
tspan = (0.0, 365.0 * 24.0 * 60.0)
# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame
# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]
t_data = collect(Float64, 1:size(data, 1))
dataloader = DataLoader([
vcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data,
t_data),
])
# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)
chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),
Flux.Dense(hidden_units, hidden_units, σ),
Flux.Dense(hidden_units, 1)) |>
Flux.f64
indvars = [t]
depvars = [:T_in, :T_ext, :Q_heating, :Q_cooling, :Q_sun, :Q_lights, :Q_equipment]
u_ = hcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data)
init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]
function additional_loss(phi, θ, p)
return sum(sum(abs2, phi[1](t_data', θ[sep[1]]) .- u_[:, 1]))
end
discretization = NeuralPDE.PhysicsInformedNN([chain1],
NeuralPDE.GridTraining(dt), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)
# callback
# Parameter Optimization
res = Optimization.solve(prob,
BFGS(),
maxiters = 1000, callback = callback)
@AstitvaAggarwal if I runn your code I get firs an error that pde_ssytem is not defined and after defining it I get:
BoundsError: attempt to access 1-element Vector{UnitRange{Int64}} at index [2]
Stacktrace:
[1] getindex(A::Vector{UnitRange{Int64}}, i1::Int64)
@ Base .\essentials.jl:13
[2] build_symbolic_loss_function(pinnrep::NeuralPDE.PINNRepresentation, eqs::Equation; eq_params::SciMLBase.NullParameters, param_estim::Bool, default_p::Nothing, bc_indvars::Vector{Any}, integrand::Nothing, dict_transformation_vars::Nothing, transformation_vars::Nothing, integrating_depvars::Vector{Symbol})
@ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:92
[3] #290
@ .\none:0 [inlined]
[4] iterate
@ .\generator.jl:47 [inlined]
[5] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Equation}, Vector{Vector{Any}}, Vector{Vector{Symbol}}}}, NeuralPDE.var"#290#304"{NeuralPDE.PINNRepresentation}})
@ Base .\array.jl:782
[6] symbolic_discretize(pde_system::PDESystem, discretization::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}}, typeof(NeuralPDE.numeric_derivative), Bool, typeof(additional_loss), Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})
@ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:542
[7] discretize(pde_system::PDESystem, discretization::PhysicsInformedNN{GridTraining{Float64}, Nothing, Vector{NeuralPDE.Phi{Optimisers.Restructure{Flux.Chain{Tuple{Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(σ), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(identity), Matrix{Float64}, Vector{Float64}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Int64, Int64, Tuple{}}}}}}}, Nothing}}, typeof(NeuralPDE.numeric_derivative), Bool, typeof(additional_loss), Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}})
@ NeuralPDE C:\Users\loren\.julia\packages\NeuralPDE\F4RlZ\src\discretize.jl:683
[8] top-level scope
@ In[2]:92
I use dt=600 because I thought that everything was in seconds so 600s=10min. Can you better explain how dt works? Why did you put chain1
to one input? I thought and wanted to use all the others as input because the affect T_in
, some of the inputs are not controllable: T_ext
(External Temperature), Q_sun
(Solar radiation), but the others are controllable loads to the system and I want the network to give me outputs with respect to those values. There is still some things that I'm not getting, I have seen that you put all the data in u_
how do you tell to the network to be trained in u_
I see it only in the additional_loss
how the equation in eqs
uses this data to solve the ode problem? How dose it know in which position of the u_
are the data?
Sorry for all of this questions but coming from python those things for me in Julia remain unclear.
hey @LorenzoBottaccioli, ignore the previous code things got mixed up, check this out, `
using NeuralPDE, Lux, ModelingToolkit, DataFrames, CSV, DataLoaders, Flux, IntervalSets,
Optimization, OptimizationOptimJL, Zygote
# Definisci il modello dell'equazione differenziale
@parameters t, R, C, Cs
@variables T_in(..)
Dt = Differential(t)
domains = [t ∈ Interval(0.0, 365.0 * 24.0 * 60.0)]
eqs = [T_in(t) ~ T_in(t)]
bcs = [Dt(T_in(0.0)) ~ 4.48]
dt = 10.0 # 600 seconds (10 minute)
# Define the temporal space
tspan = Interval(0.0, 365.0 * 24.0 * 60.0)
# load sampled data from CSV
data = CSV.File("shoebox_free.csv") |> DataFrame
# Put the sampled data in dedicated variables
T_in_data = data."OSGB1000005735772_FLOOR_1:Zone Mean Air Temperature [C](TimeStep)"[1:500]
T_ext_data = data."Environment:Site Outdoor Air Drybulb Temperature [C](TimeStep)"[1:500]
Q_heating_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Heating Rate [W](TimeStep)"[1:500]
Q_cooling_data = data."OSGB1000005735772_FLOOR_1_HVAC:Zone Ideal Loads Zone Total Cooling Rate [W](TimeStep)"[1:500]
Q_sun_data = data."Environment:Site Direct Solar Radiation Rate per Area [W/m2](TimeStep)"[1:500]
Q_lights_data = data."OSGB1000005735772_FLOOR_1:Zone Lights Total Heating Rate [W](TimeStep)"[1:500]
Q_equipment_data = data."OSGB1000005735772_FLOOR_1:Zone Electric Equipment Total Heating Rate [W](TimeStep)"[1:500]
t_data = collect(Float64, 0.0:10.0:(365.0 * 24.0 * 60.0))[1:500]
dataloader = DataLoader([
vcat(T_in_data,
T_ext_data,
Q_heating_data,
Q_cooling_data,
Q_sun_data,
Q_lights_data,
Q_equipment_data,
t_data),
])
# Define the NN
input_dim = 1
hidden_units = 8
len = length(t_data)
chain1 = Flux.Chain(Flux.Dense(input_dim, hidden_units, σ),
Flux.Dense(hidden_units, hidden_units, σ),
Flux.Dense(hidden_units, 1))
init_params = [Flux.destructure(chain1)[1]]
acum = [0; accumulate(+, length.(init_params))]
sep = [(acum[i] + 1):acum[i + 1] for i in 1:(length(acum) - 1)]
Phi = NeuralPDE.Phi(chain1)
function additional_loss(phi, θ, p)
tin = phi[1](t_data', θ[sep[1]])
R, C, Cs = p
phi_ = (p) -> Phi(p, θ[sep[1]])[1]
dphi = Zygote.gradient(phi_, t_data')
return sum(abs2, vec(tin[1, :]) .- T_in_data) + sum(sum(abs2,
(tin' .- T_ext_data) ./ (R * C) +
(Q_heating_data .- Q_cooling_data) ./ C +
Q_sun_data ./ Cs +
(Q_lights_data .+ Q_equipment_data) ./ C - dphi[1][1, :]))
end
@named pde_system = NeuralPDE.PDESystem(eqs, bcs,
domains,
[t],
[
T_in(t),
],
[R, C, Cs],
defaults = Dict([R => 1.0, C => 1.0, Cs => 1.0]))#[R, C, Cs])
discretization = NeuralPDE.PhysicsInformedNN([chain1],
NeuralPDE.GridTraining([dt]), param_estim = true, additional_loss = additional_loss)
prob = NeuralPDE.discretize(pde_system, discretization)
callback = function (p, l)
println("Current loss is: $l")
return false
end
res = Optimization.solve(prob,
BFGS(),
maxiters = 1000, callback = callback)
` Also dt should be in same units as the timespan interval(in your case minutes), the actual eq you are trying to solve is in the additional_loss function whereas the eq passed into the pde_system is trivial (bc is original). I was thinking of using Interpolations.jl for your discrete data but that just makes things longer
@LorenzoBottaccioli does this solve your issue?
Hey @LorenzoBottaccioli, now https://github.com/SciML/NeuralPDE.jl/pull/795 is merged, you can do parameter estimation with NNODE
. This is at the ODEProblem
level.
Hi I'm trying to replicare the INVERSE PROBLEM for parameter estimation on a simple ODE here the code:
Here the error I get when I run :
@named ode_system = NeuralPDE.ODESystem(eqs, [t], [T_in(t), T_ext(t), Q_heating(t), Q_cooling(t), Q_sun(t), Q_lights(t), Q_equipment(t)], [R, C, Cs])