GeorgeR227
commented
3 months ago I'm running some benchmarks and I think Decapodes by itself precompiles fairly quickly, only 7.2ms. Overall compilation time is a it over 2 seconds. The initial loading time should have also been heavily improved by #216. However I'm still looking at compilation times for our larger functions like gensim.
julia> @time_imports using Decapodes
0.5 ms Reexport
0.6 ms Compat
0.4 ms Compat → CompatLinearAlgebraExt
0.4 ms StructEquality
6.3 ms Preferences
0.5 ms PrecompileTools
0.8 ms StaticArraysCore
115.2 ms StaticArrays
49.6 ms MLStyle
0.6 ms AlgebraicInterfaces
0.4 ms DataValueInterfaces
0.7 ms DataAPI
0.4 ms IteratorInterfaceExtensions
0.4 ms TableTraits
6.5 ms Tables
4.9 ms StringManipulation
8.1 ms Crayons
0.9 ms LaTeXStrings
60.8 ms PrettyTables
5.4 ms MacroTools
0.6 ms CompTime
3.5 ms StructTypes
┌ 0.0 ms Parsers.__init__()
28.7 ms Parsers 41.74% compilation time
21.7 ms JSON3
18.5 ms DataStructures
1.5 ms Combinatorics
1.0 ms Permutations
28.8 ms ACSets
┌ 14.6 ms SuiteSparse_jll.__init__()
15.4 ms SuiteSparse_jll
┌ 6.2 ms SparseArrays.CHOLMOD.__init__() 52.06% compilation time
112.0 ms SparseArrays 2.87% compilation time
0.8 ms Statistics
0.4 ms StaticArrays → StaticArraysStatisticsExt
12.5 ms FixedPointNumbers
┌ 9.5 ms ColorTypes.__init__() 99.89% compilation time (100% recompilation)
27.4 ms ColorTypes 34.63% compilation time (100% recompilation)
25.2 ms Colors
153.6 ms GATlab
0.8 ms JLLWrappers
┌ 5.4 ms Zlib_jll.__init__()
6.1 ms Zlib_jll
┌ 4.5 ms Libiconv_jll.__init__()
5.4 ms Libiconv_jll
┌ 2.0 ms XML2_jll.__init__()
2.8 ms XML2_jll
2.2 ms LightXML
14.1 ms IterTools
2.1 ms Measures
┌ 0.0 ms Requires.__init__()
0.8 ms Requires
6.3 ms JSON
┌ 0.0 ms Compose.__init__()
7.8 ms Compose
0.8 ms PrettyPrint
0.7 ms NameResolution
2.5 ms JuliaVariables
6.9 ms GeneralizedGenerated
66.0 ms Catlab
10.8 ms DiagrammaticEquations
43.6 ms FillArrays
2.5 ms FillArrays → FillArraysSparseArraysExt
0.8 ms FillArrays → FillArraysStatisticsExt
136.0 ms ArrayLayouts
1.0 ms ArrayLayouts → ArrayLayoutsSparseArraysExt
47.0 ms LazyArrays
0.8 ms LazyArrays → LazyArraysStaticArraysExt
0.8 ms ConstructionBase
0.7 ms ConstructionBase → ConstructionBaseStaticArraysExt
16.7 ms StructArrays
1.0 ms StructArrays → StructArraysStaticArraysExt
0.9 ms StructArrays → StructArraysSparseArraysExt
1.9 ms Extents
5.1 ms GeoInterface
┌ 134.2 ms EarCut_jll.__init__() 97.00% compilation time (100% recompilation)
135.2 ms EarCut_jll 96.30% compilation time (100% recompilation)
95.8 ms GeometryBasics
12.7 ms DataMigrations
┌ 0.0 ms FileIO.__init__()
28.2 ms FileIO
5.1 ms MeshIO
10.3 ms CombinatorialSpaces
5.7 ms Krylov
12.5 ms ChainRulesCore
0.9 ms StaticArrays → StaticArraysChainRulesCoreExt
0.8 ms ChainRulesCore → ChainRulesCoreSparseArraysExt
0.7 ms SuiteSparse
1.5 ms ArrayInterface
0.9 ms ArrayInterface → ArrayInterfaceStaticArraysCoreExt
0.7 ms IfElse
12.3 ms Static
29.0 ms StaticArrayInterface
1.5 ms StaticArrayInterface → StaticArrayInterfaceStaticArraysExt
2.3 ms Functors
0.8 ms PackageExtensionCompat
┌ 0.0 ms ComponentArrays.__init__()
42.8 ms ComponentArrays
0.9 ms ComponentArrays → ComponentArraysConstructionBaseExt
4.4 ms IrrationalConstants
1.0 ms DiffRules
1.2 ms DiffResults
┌ 10.5 ms OpenLibm_jll.__init__()
11.4 ms OpenLibm_jll
0.9 ms NaNMath
┌ 0.0 ms DocStringExtensions.__init__()
1.9 ms DocStringExtensions
0.8 ms LogExpFunctions
0.8 ms LogExpFunctions → LogExpFunctionsChainRulesCoreExt
┌ 23.1 ms CompilerSupportLibraries_jll.__init__() 12.03% compilation time
24.0 ms CompilerSupportLibraries_jll 11.60% compilation time
┌ 1.8 ms OpenSpecFun_jll.__init__()
2.9 ms OpenSpecFun_jll
4.9 ms SpecialFunctions
1.0 ms SpecialFunctions → SpecialFunctionsChainRulesCoreExt
0.7 ms CommonSubexpressions
28.9 ms ForwardDiff
1.1 ms ForwardDiff → ForwardDiffStaticArraysExt
0.8 ms Adapt
0.7 ms StructArrays → StructArraysAdaptExt
0.7 ms ComponentArrays → ComponentArraysAdaptExt
0.8 ms Adapt → AdaptStaticArraysExt
1.6 ms PreallocationTools
7.3 ms Decapodes
I think we can use the techniques in this post to bring down Decapodes precompile times
https://sciml.ai/news/2022/09/21/compile_time/#starting_the_process_profiling_why_ordinarydiffeq_first_solve_time_was_30_second_compilation
SnoopCompile should also help find type inference failures, which were the cause of the major performance problems in ACSets.jl that were biting Decapodes.