search

go-hep / hep

hep is the mono repository holding all of go-hep.org/x/hep packages and tools
https://go-hep.org
BSD 3-Clause "New" or "Revised" License
230 stars 35 forks source link

cmd/root2npy: improve memory usage #902

Closed sbinet closed 2 years ago

sbinet commented 2 years ago

This CL modifies root2npy to process Trees column by column so only 1 numpy buffer is materialized in-memory.

$> benchstat ./ref.txt ./new.txt
name       old time/op    new time/op    delta
Process-8    2.48ms ± 1%    2.39ms ± 2%   -3.44%  (p=0.000 n=28+29)

name       old alloc/op   new alloc/op   delta
Process-8    3.50MB ± 1%    2.64MB ± 2%  -24.63%  (p=0.000 n=30+29)

name       old allocs/op  new allocs/op  delta
Process-8     3.41k ± 0%     3.28k ± 0%   -3.60%  (p=0.000 n=30+30)
sbinet commented 2 years ago

PTAL @Jatharachill

codecov-commenter commented 2 years ago

Codecov Report

Merging #902 (37a9ac0) into main (04a2d5f) will decrease coverage by 0.00%. The diff coverage is 63.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #902      +/-   ##
==========================================
- Coverage   73.90%   73.89%   -0.01%     
==========================================
  Files         400      400              
  Lines       48403    48412       +9     
==========================================
+ Hits        35772    35776       +4     
- Misses      10402    10406       +4     
- Partials     2229     2230       +1
Impacted Files Coverage Δ
cmd/root2npy/main.go 57.29% <63.63%> (-1.33%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 04a2d5f...37a9ac0. Read the comment docs.