SymbolicML / DynamicExpressions.jl

Ridiculously fast symbolic expressions
https://symbolicml.org/DynamicExpressions.jl/dev
Apache License 2.0
90 stars 11 forks source link

Type stable wrapping of Zygote gradients #49

Closed MilesCranmer closed 10 months ago

MilesCranmer commented 10 months ago

Zygote gradients return nothing when a variable is not used. For example, in a branch of an operator which does not use a variable.

This PR implements a fix for this behavior to ensure type stability.

github-actions[bot] commented 10 months ago

Benchmark Results

master b3328774db3226... t[master]/t[b3328774db3226...]
eval/ComplexF32/evaluation 13.5 ± 1.1 ms 13.4 ± 1 ms 1.01
eval/ComplexF64/evaluation 17 ± 1.4 ms 17 ± 1.6 ms 1
eval/Float32/derivative 26.4 ± 4.8 ms 25 ± 3.7 ms 1.06
eval/Float32/derivative_turbo 25.5 ± 3.9 ms 25.5 ± 3.7 ms 0.998
eval/Float32/evaluation 4.71 ± 0.43 ms 4.62 ± 0.45 ms 1.02
eval/Float32/evaluation_turbo 1.49 ± 0.16 ms 1.48 ± 0.15 ms 1
eval/Float64/derivative 0.0354 ± 0.0016 s 0.0344 ± 0.0016 s 1.03
eval/Float64/derivative_turbo 0.0388 ± 0.0019 s 0.0384 ± 0.0016 s 1.01
eval/Float64/evaluation 5.12 ± 0.52 ms 5.01 ± 0.48 ms 1.02
eval/Float64/evaluation_turbo 2.75 ± 0.25 ms 2.74 ± 0.27 ms 1.01
time_to_load 1.11 ± 0.036 s 1.14 ± 0.023 s 0.972
utils/combine_operators 0.0703 ± 0.014 ms 0.0684 ± 0.012 ms 1.03
utils/convert/break_sharing 0.0662 ± 0.014 ms 0.064 ± 0.018 ms 1.03
utils/convert/preserve_sharing 0.279 ± 0.043 ms 0.272 ± 0.059 ms 1.03
utils/copy/break_sharing 0.0607 ± 0.016 ms 0.0601 ± 0.016 ms 1.01
utils/copy/preserve_sharing 0.27 ± 0.049 ms 0.263 ± 0.052 ms 1.03
utils/count_constants 28.9 ± 5 μs 29.9 ± 5.4 μs 0.967
utils/count_depth 0.0351 ± 0.0063 ms 0.0335 ± 0.0056 ms 1.05
utils/count_nodes 29.4 ± 6.8 μs 29.2 ± 5.1 μs 1.01
utils/get_set_constants! 0.111 ± 0.019 ms 0.111 ± 0.019 ms 1
utils/has_constants 10.3 ± 1.9 μs 9.48 ± 1.8 μs 1.08
utils/has_operators 2.97 ± 0.37 μs 2.62 ± 0.31 μs 1.13
utils/index_constants 0.0911 ± 0.017 ms 0.0904 ± 0.018 ms 1.01
utils/is_constant 10.1 ± 1.8 μs 10.1 ± 1.8 μs 0.999
utils/simplify_tree 0.215 ± 0.035 ms 0.213 ± 0.041 ms 1.01
utils/string_tree 0.445 ± 0.079 ms 0.427 ± 0.072 ms 1.04

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).