Closed JKRT closed 6 months ago
ChrisRackauckas
commented
2 years ago Thanks for the test case. @YingboMa has been looking into compile time growth (and expanding the use of symbolic arrays) and can use this as a test case. Julia v1.8 (unreleased) has numerous changes as well to accommodate some compile time fixes here.
JKRT
commented
2 years ago @ChrisRackauckas Thanks,
I did some testing for Julia 1.9-DEV this is the numbers I got.
A small little script below, what this does is to compile the MTK model to machine code in memory (but it does not execute)
using Revise
import OMFrontend
import OMBackend
function runTest(filePath, modelName)
@info "Trying to run $modelName through the pipeline"
@info "Time of generating the AST"
@time ast = OMFrontend.parseFile("./$(filePath)")
@info "Time of generating SCode"
@time scodeProgram = OMFrontend.translateToSCode(ast)
@info "Time to flat DAE"
@time (dae, cache) = OMFrontend.instantiateSCodeToDAE(modelName, scodeProgram)
# Base.dump(dae)
# @info dae
#= Translate and load it into our environment =#
@info "Time of backend transformation"
@time OMBackend.translate(dae; BackendMode = OMBackend.MTK_MODE)
#= Simulate using the backend =#
# @info "Time of simulation"
# Do not execute MTK yet
# @time res = OMBackend.simulateModel(modelName, tspan = (0.0, 1.0));
return 0
# OMBackend.plot(res)
end
function writeCascToFile(path, model)
runTest(path, model)
OMBackend.writeModelToFile(model, "./CascModels/$(model).txt"; keepComments = false, formatFile = true, mode = OMBackend.MTK_MODE)
println("Done writing to file")
end
println("Casc 10:")
writeCascToFile("./Models/Casc10.mo", "Casc10")
println("Casc 100:")
writeCascToFile("./Models/Casc100.mo", "Casc100")
println("Casc 200:")
writeCascToFile("./Models/Casc200.mo", "Casc200")
println("Casc 400:")
writeCascToFile("./Models/Casc400.mo", "Casc400")
println("Casc 800:")
writeCascToFile("./Models/Casc800.mo", "Casc800")
println("Casc 1600:")
writeCascToFile("./Models/Casc1600.mo", "Casc1600")
println("Casc 3200:")
writeCascToFile("./Models/Casc3200.mo", "Casc3200")
println("Casc 6400:")
writeCascToFile("./Models/Casc6400.mo", "Casc6400")In this test I generate MTK models up to 6400 equations and variables for the example
Total time for doing this (unscientific measurements using time once on my laptop)
Julia 1.7.1:
159.302903 seconds (1.21 G allocations: 40.454 GiB, 8.14% gc time, 0.06% compilation time)
Julia 1.9-DEV:
140.535181 seconds (1.21 G allocations: 40.389 GiB, 6.74% gc time, 0.06% compilation time)
So I manage to compile the models a little a bit faster (roughly 13%) on Julia 1.9 dev. However, if I try to run and simulate the results are a bit different
Julia 1.7.1:
Casc10:
[ Info: Time of simulation
0.289369 seconds (455.41 k allocations: 26.606 MiB, 94.41% compilation time)
Casc100:
[ Info: Time of simulation
1.539921 seconds (1.36 M allocations: 71.389 MiB, 93.58% compilation time)
Casc200:
[ Info: Time of simulation
3.818414 seconds (2.40 M allocations: 127.000 MiB, 96.15% compilation time)
Casc400:
[ Info: Time of simulation
17.513228 seconds (4.48 M allocations: 256.559 MiB, 96.44% compilation time)
Casc800:
[ Info: Time of simulation
92.846937 seconds (8.57 M allocations: 583.215 MiB, 0.16% gc time, 94.78% compilation time)Julia 1.9-DEV:
Casc10:
[ Info: Time of simulation
0.383642 seconds (803.45 k allocations: 42.876 MiB, 95.80% compilation time)
Casc100:
[ Info: Time of simulation
2.853880 seconds (3.43 M allocations: 179.410 MiB, 5.79% gc time, 96.34% compilation time)
Casc200:
[ Info: Time of simulation
6.223935 seconds (6.39 M allocations: 340.632 MiB, 95.81% compilation time)
Casc400:
[ Info: Time of simulation
40.227598 seconds (29.13 M allocations: 1.704 GiB, 1.30% gc time, 98.32% compilation time)
[ Info: Time of simulation
163.416695 seconds (50.70 M allocations: 3.303 GiB, 1.11% gc time, 94.46% compilation time)It seems the MTK part of running the code in 1.9-DEV is a regression, however, the compilation time (from the Julia side seems to have improved in 1.9)
(I will include some more models to play around with, I think these really huge functions are one issue (So what I would need is some way of constructing a function gradually before creating a system), however, compiling them in Julia is not that bad in the first experiment so I will try to reduce them by smarter construction)
I also included some more Casc models here (Although for this test I did not go beyond 800) Casc3200.txt Casc6400.txt
Cheers, John
JKRT
commented
2 years ago This is starting to become like a log book.
I did some experimentation in my compiler by abusing MTK just a little bit 🧷 Instead of generating all equations and variables directly I instead decompose the system using inner functions.
I am not sure how MTK works internally but I guess having a similar scheme as a part of MTK would improve base performance somewhat for people who are writing equation-object-oriented code (That have also reported slowdowns such as #1286), that would also make my code generator simpler
Some current timings:
Casc10:
[ Info: Time of simulation
0.239821 seconds (544.30 k allocations: 33.200 MiB, 93.25% compilation time)
Casc100
[ Info: Time of simulation
0.648164 seconds (832.73 k allocations: 46.834 MiB, 80.84% compilation time)
Casc200
[ Info: Time of simulation
1.241312 seconds (1.19 M allocations: 64.276 MiB, 9.33% gc time, 82.89% compilation time)
Casc400
[ Info: Time of simulation
2.394954 seconds (1.92 M allocations: 101.958 MiB, 72.05% compilation time)
Casc800
[ Info: Time of simulation
9.166931 seconds (3.34 M allocations: 187.152 MiB, 1.32% gc time, 38.73% compilation time)This technique I applied in a rather naive fashion but I managed to improve compilation time I guess about 10X! 🕵️
Example of a model composed in this way (Note a bit of hacking was involved)
(Warning long list of code)
using ModelingToolkit
using DifferentialEquations
begin
begin
saved_values_Casc100 = SavedValues(Float64, Tuple{Float64,Array})
function Casc100CallbackSet(aux)
local p = aux[1]
local reals = aux[2]
nothing
return CallbackSet()
end
end
function Casc100Model(tspan = (0.0, 1.0))
@variables t #= C:\Users\johti17\Projects\Programming\JuliaPackages\OM.jl\OMBackend.jl\src\CodeGeneration\MTK_CodeGeneration.jl:219 =#
parameters = ModelingToolkit.@parameters((N, tau, T)) #= C:\Users\johti17\Projects\Programming\JuliaPackages\OM.jl\OMBackend.jl\src\CodeGeneration\MTK_CodeGeneration.jl:220 =#
begin
variableConstructors = Function[]
begin
function generateStateVariables1()
(
:t,
:(x_1(t)),
:(x_2(t)),
:(x_3(t)),
:(x_4(t)),
:(x_5(t)),
:(x_6(t)),
:(x_7(t)),
:(x_8(t)),
:(x_9(t)),
:(x_10(t)),
:(x_11(t)),
:(x_12(t)),
:(x_13(t)),
:(x_14(t)),
:(x_15(t)),
:(x_16(t)),
:(x_17(t)),
:(x_18(t)),
:(x_19(t)),
:(x_20(t)),
:(x_21(t)),
:(x_22(t)),
:(x_23(t)),
:(x_24(t)),
:(x_25(t)),
:(x_26(t)),
:(x_27(t)),
:(x_28(t)),
:(x_29(t)),
:(x_30(t)),
:(x_31(t)),
:(x_32(t)),
:(x_33(t)),
:(x_34(t)),
:(x_35(t)),
:(x_36(t)),
:(x_37(t)),
:(x_38(t)),
:(x_39(t)),
:(x_40(t)),
:(x_41(t)),
:(x_42(t)),
:(x_43(t)),
:(x_44(t)),
:(x_45(t)),
:(x_46(t)),
:(x_47(t)),
:(x_48(t)),
:(x_49(t)),
:(x_50(t)),
)
end
push!(variableConstructors, generateStateVariables1)
end
begin
function generateStateVariables2()
(
:t,
:(x_51(t)),
:(x_52(t)),
:(x_53(t)),
:(x_54(t)),
:(x_55(t)),
:(x_56(t)),
:(x_57(t)),
:(x_58(t)),
:(x_59(t)),
:(x_60(t)),
:(x_61(t)),
:(x_62(t)),
:(x_63(t)),
:(x_64(t)),
:(x_65(t)),
:(x_66(t)),
:(x_67(t)),
:(x_68(t)),
:(x_69(t)),
:(x_70(t)),
:(x_71(t)),
:(x_72(t)),
:(x_73(t)),
:(x_74(t)),
:(x_75(t)),
:(x_76(t)),
:(x_77(t)),
:(x_78(t)),
:(x_79(t)),
:(x_80(t)),
:(x_81(t)),
:(x_82(t)),
:(x_83(t)),
:(x_84(t)),
:(x_85(t)),
:(x_86(t)),
:(x_87(t)),
:(x_88(t)),
:(x_89(t)),
:(x_90(t)),
:(x_91(t)),
:(x_92(t)),
:(x_93(t)),
:(x_94(t)),
:(x_95(t)),
:(x_96(t)),
:(x_97(t)),
:(x_98(t)),
:(x_99(t)),
:(x_100(t)),
)
end
push!(variableConstructors, generateStateVariables2)
end
end
componentVars = []
for constructor in variableConstructors
res = eval(
ModelingToolkit.Symbolics._parse_vars(
"CustomCall",
Real,
constructor(),
),
)
push!(componentVars, res[2:end])
end
vars = collect(Iterators.flatten(componentVars))
der = Differential(t)
equationComponents = []
begin
equationConstructors = Function[]
begin
function generateEquations0()
[
der(x_1) ~ (1.0 - x_1) / tau,
der(x_2) ~ (x_1 - x_2) / tau,
der(x_3) ~ (x_2 - x_3) / tau,
der(x_4) ~ (x_3 - x_4) / tau,
der(x_5) ~ (x_4 - x_5) / tau,
der(x_6) ~ (x_5 - x_6) / tau,
der(x_7) ~ (x_6 - x_7) / tau,
der(x_8) ~ (x_7 - x_8) / tau,
der(x_9) ~ (x_8 - x_9) / tau,
der(x_10) ~ (x_9 - x_10) / tau,
der(x_11) ~ (x_10 - x_11) / tau,
der(x_12) ~ (x_11 - x_12) / tau,
der(x_13) ~ (x_12 - x_13) / tau,
der(x_14) ~ (x_13 - x_14) / tau,
der(x_15) ~ (x_14 - x_15) / tau,
der(x_16) ~ (x_15 - x_16) / tau,
der(x_17) ~ (x_16 - x_17) / tau,
der(x_18) ~ (x_17 - x_18) / tau,
der(x_19) ~ (x_18 - x_19) / tau,
der(x_20) ~ (x_19 - x_20) / tau,
der(x_21) ~ (x_20 - x_21) / tau,
der(x_22) ~ (x_21 - x_22) / tau,
der(x_23) ~ (x_22 - x_23) / tau,
der(x_24) ~ (x_23 - x_24) / tau,
der(x_25) ~ (x_24 - x_25) / tau,
der(x_26) ~ (x_25 - x_26) / tau,
der(x_27) ~ (x_26 - x_27) / tau,
der(x_28) ~ (x_27 - x_28) / tau,
der(x_29) ~ (x_28 - x_29) / tau,
der(x_30) ~ (x_29 - x_30) / tau,
der(x_31) ~ (x_30 - x_31) / tau,
der(x_32) ~ (x_31 - x_32) / tau,
der(x_33) ~ (x_32 - x_33) / tau,
der(x_34) ~ (x_33 - x_34) / tau,
der(x_35) ~ (x_34 - x_35) / tau,
der(x_36) ~ (x_35 - x_36) / tau,
der(x_37) ~ (x_36 - x_37) / tau,
der(x_38) ~ (x_37 - x_38) / tau,
der(x_39) ~ (x_38 - x_39) / tau,
der(x_40) ~ (x_39 - x_40) / tau,
der(x_41) ~ (x_40 - x_41) / tau,
der(x_42) ~ (x_41 - x_42) / tau,
der(x_43) ~ (x_42 - x_43) / tau,
der(x_44) ~ (x_43 - x_44) / tau,
der(x_45) ~ (x_44 - x_45) / tau,
der(x_46) ~ (x_45 - x_46) / tau,
der(x_47) ~ (x_46 - x_47) / tau,
der(x_48) ~ (x_47 - x_48) / tau,
der(x_49) ~ (x_48 - x_49) / tau,
der(x_50) ~ (x_49 - x_50) / tau,
]
end
push!(equationConstructors, generateEquations0)
end
begin
function generateEquations1()
[
der(x_51) ~ (x_50 - x_51) / tau,
der(x_52) ~ (x_51 - x_52) / tau,
der(x_53) ~ (x_52 - x_53) / tau,
der(x_54) ~ (x_53 - x_54) / tau,
der(x_55) ~ (x_54 - x_55) / tau,
der(x_56) ~ (x_55 - x_56) / tau,
der(x_57) ~ (x_56 - x_57) / tau,
der(x_58) ~ (x_57 - x_58) / tau,
der(x_59) ~ (x_58 - x_59) / tau,
der(x_60) ~ (x_59 - x_60) / tau,
der(x_61) ~ (x_60 - x_61) / tau,
der(x_62) ~ (x_61 - x_62) / tau,
der(x_63) ~ (x_62 - x_63) / tau,
der(x_64) ~ (x_63 - x_64) / tau,
der(x_65) ~ (x_64 - x_65) / tau,
der(x_66) ~ (x_65 - x_66) / tau,
der(x_67) ~ (x_66 - x_67) / tau,
der(x_68) ~ (x_67 - x_68) / tau,
der(x_69) ~ (x_68 - x_69) / tau,
der(x_70) ~ (x_69 - x_70) / tau,
der(x_71) ~ (x_70 - x_71) / tau,
der(x_72) ~ (x_71 - x_72) / tau,
der(x_73) ~ (x_72 - x_73) / tau,
der(x_74) ~ (x_73 - x_74) / tau,
der(x_75) ~ (x_74 - x_75) / tau,
der(x_76) ~ (x_75 - x_76) / tau,
der(x_77) ~ (x_76 - x_77) / tau,
der(x_78) ~ (x_77 - x_78) / tau,
der(x_79) ~ (x_78 - x_79) / tau,
der(x_80) ~ (x_79 - x_80) / tau,
der(x_81) ~ (x_80 - x_81) / tau,
der(x_82) ~ (x_81 - x_82) / tau,
der(x_83) ~ (x_82 - x_83) / tau,
der(x_84) ~ (x_83 - x_84) / tau,
der(x_85) ~ (x_84 - x_85) / tau,
der(x_86) ~ (x_85 - x_86) / tau,
der(x_87) ~ (x_86 - x_87) / tau,
der(x_88) ~ (x_87 - x_88) / tau,
der(x_89) ~ (x_88 - x_89) / tau,
der(x_90) ~ (x_89 - x_90) / tau,
der(x_91) ~ (x_90 - x_91) / tau,
der(x_92) ~ (x_91 - x_92) / tau,
der(x_93) ~ (x_92 - x_93) / tau,
der(x_94) ~ (x_93 - x_94) / tau,
der(x_95) ~ (x_94 - x_95) / tau,
der(x_96) ~ (x_95 - x_96) / tau,
der(x_97) ~ (x_96 - x_97) / tau,
der(x_98) ~ (x_97 - x_98) / tau,
der(x_99) ~ (x_98 - x_99) / tau,
der(x_100) ~ (x_99 - x_100) / tau,
]
end
push!(equationConstructors, generateEquations1)
end
end
for constructor in equationConstructors
push!(equationComponents, constructor())
end
eqs = collect(Iterators.flatten(equationComponents))
nonLinearSystem = ModelingToolkit.ODESystem(
eqs,
t,
vars,
parameters,
name = :($(Symbol("Casc100"))),
)
pars = Dict(N => float(100), tau => float(T / 100.0), T => float(1.0))
initialValues = [
x_1 => 0.0,
x_2 => 0.0,
x_3 => 0.0,
x_4 => 0.0,
x_5 => 0.0,
x_6 => 0.0,
x_7 => 0.0,
x_8 => 0.0,
x_9 => 0.0,
x_10 => 0.0,
x_11 => 0.0,
x_12 => 0.0,
x_13 => 0.0,
x_14 => 0.0,
x_15 => 0.0,
x_16 => 0.0,
x_17 => 0.0,
x_18 => 0.0,
x_19 => 0.0,
x_20 => 0.0,
x_21 => 0.0,
x_22 => 0.0,
x_23 => 0.0,
x_24 => 0.0,
x_25 => 0.0,
x_26 => 0.0,
x_27 => 0.0,
x_28 => 0.0,
x_29 => 0.0,
x_30 => 0.0,
x_31 => 0.0,
x_32 => 0.0,
x_33 => 0.0,
x_34 => 0.0,
x_35 => 0.0,
x_36 => 0.0,
x_37 => 0.0,
x_38 => 0.0,
x_39 => 0.0,
x_40 => 0.0,
x_41 => 0.0,
x_42 => 0.0,
x_43 => 0.0,
x_44 => 0.0,
x_45 => 0.0,
x_46 => 0.0,
x_47 => 0.0,
x_48 => 0.0,
x_49 => 0.0,
x_50 => 0.0,
x_51 => 0.0,
x_52 => 0.0,
x_53 => 0.0,
x_54 => 0.0,
x_55 => 0.0,
x_56 => 0.0,
x_57 => 0.0,
x_58 => 0.0,
x_59 => 0.0,
x_60 => 0.0,
x_61 => 0.0,
x_62 => 0.0,
x_63 => 0.0,
x_64 => 0.0,
x_65 => 0.0,
x_66 => 0.0,
x_67 => 0.0,
x_68 => 0.0,
x_69 => 0.0,
x_70 => 0.0,
x_71 => 0.0,
x_72 => 0.0,
x_73 => 0.0,
x_74 => 0.0,
x_75 => 0.0,
x_76 => 0.0,
x_77 => 0.0,
x_78 => 0.0,
x_79 => 0.0,
x_80 => 0.0,
x_81 => 0.0,
x_82 => 0.0,
x_83 => 0.0,
x_84 => 0.0,
x_85 => 0.0,
x_86 => 0.0,
x_87 => 0.0,
x_88 => 0.0,
x_89 => 0.0,
x_90 => 0.0,
x_91 => 0.0,
x_92 => 0.0,
x_93 => 0.0,
x_94 => 0.0,
x_95 => 0.0,
x_96 => 0.0,
x_97 => 0.0,
x_98 => 0.0,
x_99 => 0.0,
x_100 => 0.0,
]
firstOrderSystem = ModelingToolkit.ode_order_lowering(nonLinearSystem)
reducedSystem = firstOrderSystem
local event_p = [100, 0, 1.0]
local discreteVars = collect(values(Dict([])))
local event_vars = vcat(
collect(
values(
Dict([
x_1 => 0.0,
x_2 => 0.0,
x_3 => 0.0,
x_4 => 0.0,
x_5 => 0.0,
x_6 => 0.0,
x_7 => 0.0,
x_8 => 0.0,
x_9 => 0.0,
x_10 => 0.0,
x_11 => 0.0,
x_12 => 0.0,
x_13 => 0.0,
x_14 => 0.0,
x_15 => 0.0,
x_16 => 0.0,
x_17 => 0.0,
x_18 => 0.0,
x_19 => 0.0,
x_20 => 0.0,
x_21 => 0.0,
x_22 => 0.0,
x_23 => 0.0,
x_24 => 0.0,
x_25 => 0.0,
x_26 => 0.0,
x_27 => 0.0,
x_28 => 0.0,
x_29 => 0.0,
x_30 => 0.0,
x_31 => 0.0,
x_32 => 0.0,
x_33 => 0.0,
x_34 => 0.0,
x_35 => 0.0,
x_36 => 0.0,
x_37 => 0.0,
x_38 => 0.0,
x_39 => 0.0,
x_40 => 0.0,
x_41 => 0.0,
x_42 => 0.0,
x_43 => 0.0,
x_44 => 0.0,
x_45 => 0.0,
x_46 => 0.0,
x_47 => 0.0,
x_48 => 0.0,
x_49 => 0.0,
x_50 => 0.0,
x_51 => 0.0,
x_52 => 0.0,
x_53 => 0.0,
x_54 => 0.0,
x_55 => 0.0,
x_56 => 0.0,
x_57 => 0.0,
x_58 => 0.0,
x_59 => 0.0,
x_60 => 0.0,
x_61 => 0.0,
x_62 => 0.0,
x_63 => 0.0,
x_64 => 0.0,
x_65 => 0.0,
x_66 => 0.0,
x_67 => 0.0,
x_68 => 0.0,
x_69 => 0.0,
x_70 => 0.0,
x_71 => 0.0,
x_72 => 0.0,
x_73 => 0.0,
x_74 => 0.0,
x_75 => 0.0,
x_76 => 0.0,
x_77 => 0.0,
x_78 => 0.0,
x_79 => 0.0,
x_80 => 0.0,
x_81 => 0.0,
x_82 => 0.0,
x_83 => 0.0,
x_84 => 0.0,
x_85 => 0.0,
x_86 => 0.0,
x_87 => 0.0,
x_88 => 0.0,
x_89 => 0.0,
x_90 => 0.0,
x_91 => 0.0,
x_92 => 0.0,
x_93 => 0.0,
x_94 => 0.0,
x_95 => 0.0,
x_96 => 0.0,
x_97 => 0.0,
x_98 => 0.0,
x_99 => 0.0,
x_100 => 0.0,
]),
),
),
discreteVars,
)
local aux = Array{Array{Float64}}(undef, 2)
aux[1] = event_p
aux[2] = event_vars
problem = ModelingToolkit.ODEProblem(
reducedSystem,
initialValues,
tspan,
pars,
callback = Casc100CallbackSet(aux),
)
return (problem, initialValues, reducedSystem, tspan, pars, vars)
end
end
(Casc100Model_problem, _, _, _, _, _) = Casc100Model()
function Casc100Simulate(tspan)
solve(Casc100Model_problem, tspan = tspan)
end
function Casc100Simulate(tspan = (0.0, 1.0); solver = Rodas5())
solve(Casc100Model_problem, tspan = tspan, solver)
endStill, issues with growing compilation although far from as severe as before. Some new models are included below (as .txt files) from 10 to 1600. Casc10Experimental.txt Casc100Experimental.txt Casc200Experimental.txt Casc400Experimental.txt Casc800Experimental.txt Casc1600Experimental.txt
Seems som of the lag observed when I wrote this post was that I used Rodas instead of AutoVern.. Now my main issue seem to be that I encounter StackOverflow when I try to get 6400 variables and equations working
ChrisRackauckas
commented
6 months ago JuliaSimCompiler is the answer.
Hi,
I would like to start by apologizing for a long text...
I have the following models, note these models are automatically generated based on different parameters to the following Modelica model
The corresponding MTK model for
N = 10(Take note that this code is automatically generated) is:A sample current timings (On my laptop) MTK compilation + Simulation
Sorry for a long post, I suppose a related issue is #1286 and #1285
Models are attached for reference. I uploaded them as txt files since the GitHub issue API do not seem to support .jl files Casc10.txt Casc100.txt Casc200.txt Casc400.txt Casc800.txt Casc1600.txt
It also takes quite some time just including the models, say that I have a model like the ones attached, how could I decompose the system better in order to reduce the compilation time overhead?
I suppose splitting the declarations of variables into separate functions might do something? See https://github.com/JuliaLang/julia/issues/19158
But then the question is how could I go about splitting the declaration of variables and equations to built it up in parts?
Say that I want to register 100 variables at the time using @variables
(Feel free to rework these Julia models so that they look more idiomatic and use them for benchmarking)
Cheers, John 🚀