rejuvyesh / PyCallChainRules.jl

Differentiate python calls from Julia
MIT License
56 stars 2 forks source link

Add benchmarks to track performance #7

Closed rejuvyesh closed 2 years ago

rejuvyesh commented 2 years ago
julia> (results["pytorchhub"]["bs=1"]["backward"]["functorch"])
BenchmarkTools.Trial: 20 samples with 1 evaluation.
 Range (min … max):  20.868 ms … 24.872 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     22.527 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   22.565 ms ±  1.328 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

   ▃▃                     ▃              █                     
  ▇██▁▁▁▁▁▁▇▇▇▁▁▁▁▁▁▁▁▇▁▁▁█▇▁▁▁▁▁▁▁▁▁▁▁▁▇█▁▁▁▁▇▁▁▁▁▁▇▁▁▁▇▁▁▁▇ ▁
  20.9 ms         Histogram: frequency by time        24.9 ms <

 Memory estimate: 6.55 KiB, allocs estimate: 26.

julia> (results["pytorchhub"]["bs=1"]["backward"]["torch"])
BenchmarkTools.Trial: 20 samples with 1 evaluation.
 Range (min … max):  50.621 ms … 185.750 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     56.413 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   67.238 ms ±  33.660 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▂█▅ ▂                                                         
  ███▅█▅▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅ ▁
  50.6 ms         Histogram: frequency by time          186 ms <

 Memory estimate: 21.44 KiB, allocs estimate: 338.

julia> (results["pytorchhub"]["bs=1"]["backward"]["jl"])
BenchmarkTools.Trial: 20 samples with 1 evaluation.
 Range (min … max):  212.245 ms … 337.308 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     222.760 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   238.807 ms ±  36.581 ms  ┊ GC (mean ± σ):  0.28% ± 0.67%

  ▃ █▃▃  ▃                                                       
  █▇███▇▇█▁▁▇▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇▁▁▁▁▁▁▁▁▁▇▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇ ▁
  212 ms           Histogram: frequency by time          337 ms <

 Memory estimate: 46.05 MiB, allocs estimate: 5854.