JuliaCollections / IterTools.jl

Common functional iterator patterns
Other
153 stars 29 forks source link

Add FieldValues iterator #46

Closed iamed2 closed 5 years ago

iamed2 commented 5 years ago

It's like (getfield(x, i) for i in 1:nfields(x)) but faster.

I considered making it a lazy array type instead but that was slower and I didn't feel like digging into why.

codecov-io commented 5 years ago

Codecov Report

Merging #46 into master will increase coverage by 0.47%. The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #46      +/-   ##
==========================================
+ Coverage   65.31%   65.78%   +0.47%     
==========================================
  Files           1        1              
  Lines         222      228       +6     
==========================================
+ Hits          145      150       +5     
- Misses         77       78       +1
Impacted Files Coverage Δ
src/IterTools.jl 65.78% <83.33%> (+0.47%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a2e4d09...43175bb. Read the comment docs.

iamed2 commented 5 years ago

Benchmarks:

julia> using IterTools, BenchmarkTools

julia> cn = 1 + 2im
1 + 2im

julia> fv_gen(x) = (getfield(x, i) for i in 1:nfields(x))
fv_gen (generic function with 1 method)

julia> fv1 = fv_gen(cn)
Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##3#4")){Complex{Int64}}}(getfield(Main, Symbol("##3#4")){Complex{Int64}}(1 + 2im), 1:2)

julia> fv2 = fieldvalues(cn)
IterTools.FieldValues{Complex{Int64}}(1 + 2im)

julia> @benchmark collect($fv1)
BenchmarkTools.Trial:
  memory estimate:  96 bytes
  allocs estimate:  1
  --------------
  minimum time:     31.497 ns (0.00% GC)
  median time:      32.952 ns (0.00% GC)
  mean time:        44.549 ns (16.53% GC)
  maximum time:     42.877 μs (99.87% GC)
  --------------
  samples:          10000
  evals/sample:     989

julia> @benchmark collect($fv2)
BenchmarkTools.Trial:
  memory estimate:  96 bytes
  allocs estimate:  1
  --------------
  minimum time:     37.041 ns (0.00% GC)
  median time:      39.687 ns (0.00% GC)
  mean time:        52.321 ns (15.29% GC)
  maximum time:     44.472 μs (99.86% GC)
  --------------
  samples:          10000
  evals/sample:     985

julia> tp = ("", 1, 2)
("", 1, 2)

julia> fv1 = fv_gen(tp)
Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##3#4")){Tuple{String,Int64,Int64}}}(getfield(Main, Symbol("##3#4")){Tuple{String,Int64,Int64}}(("", 1, 2)), 1:3)

julia> fv2 = fieldvalues(tp)
IterTools.FieldValues{Tuple{String,Int64,Int64}}(("", 1, 2))

julia> @benchmark collect($fv1)
BenchmarkTools.Trial:
  memory estimate:  256 bytes
  allocs estimate:  3
  --------------
  minimum time:     237.611 ns (0.00% GC)
  median time:      240.175 ns (0.00% GC)
  mean time:        278.074 ns (7.60% GC)
  maximum time:     106.225 μs (99.65% GC)
  --------------
  samples:          10000
  evals/sample:     401

julia> @benchmark collect($fv2)
BenchmarkTools.Trial:
  memory estimate:  112 bytes
  allocs estimate:  1
  --------------
  minimum time:     79.278 ns (0.00% GC)
  median time:      82.236 ns (0.00% GC)
  mean time:        99.955 ns (9.51% GC)
  maximum time:     48.112 μs (99.73% GC)
  --------------
  samples:          10000
  evals/sample:     918

julia> struct A
           x
           y
       end

julia> a = A(1, 2.0)
A(1, 2.0)

julia> fv1 = fv_gen(a)
Base.Generator{UnitRange{Int64},getfield(Main, Symbol("##3#4")){A}}(getfield(Main, Symbol("##3#4")){A}(A(1, 2.0)), 1:2)

julia> fv2 = fieldvalues(a)
IterTools.FieldValues{A}(A(1, 2.0))

julia> @benchmark collect($fv1)
BenchmarkTools.Trial:
  memory estimate:  240 bytes
  allocs estimate:  4
  --------------
  minimum time:     3.557 μs (0.00% GC)
  median time:      4.352 μs (0.00% GC)
  mean time:        5.236 μs (2.37% GC)
  maximum time:     1.250 ms (99.23% GC)
  --------------
  samples:          10000
  evals/sample:     8

julia> @benchmark collect($fv2)
BenchmarkTools.Trial:
  memory estimate:  96 bytes
  allocs estimate:  1
  --------------
  minimum time:     35.591 ns (0.00% GC)
  median time:      36.180 ns (0.00% GC)
  mean time:        46.157 ns (15.66% GC)
  maximum time:     35.454 μs (99.87% GC)
  --------------
  samples:          10000
  evals/sample:     993

I believe the advantage comes from FieldValues having eltype of Any, while the generator has EltypeUnknown.