JuliaFolds / Transducers.jl

Efficient transducers for Julia
https://juliafolds.github.io/Transducers.jl/dev/
MIT License
433 stars 24 forks source link

Improve performance of `collect` on size unstable collections. #554

Open MasonProtter opened 1 year ago

MasonProtter commented 1 year ago

Okay, so the low hanging fruit of SizeStable collections should be dealt with in https://github.com/JuliaFolds/Transducers.jl/pull/553, but SizeChanging collections also need attention:

julia> let v = randn(10000)
           @btime filter(x -> x < 1000, $v)
           @btime collect(Filter(x -> x < 1000), $v)
       end
  4.767 μs (2 allocations: 78.17 KiB)
  7.820 μs (10 allocations: 326.61 KiB)

Perhaps the most catastrophic offender I've seen is Partiton.

julia> let v = rand(10000), sz = 6
           @btime collect(Iterators.partition($v, $sz))
           @btime collect(Partition($sz), $v)
       end;
  3.197 μs (2 allocations: 65.17 KiB)
  18.825 ms (189965 allocations: 6.52 MiB)