JuliaPsychometrics / ItemResponseFunctions.jl

A lightweight julia package providing basic implementations of item response models
MIT License
1 stars 0 forks source link

Performance hints for polytomous models #32

Open p-gw opened 5 months ago

p-gw commented 5 months ago

Almost all functions require a call to merge_pars because low-level functions are only implemented for the most general cases.

For polytomous models the deepcopy call in merge_pars requires allocations if a normal vector of thresholds is passed. Other variants are far more performant, e.g. Tuple or StaticVector:

Vector:

beta = (a = 1.0, b = 0.2, t = [0.0, 0.2, 1.0])
@benchmark irf(GPCM, $0.0, $beta)

BenchmarkTools.Trial: 10000 samples with 10 evaluations.
 Range (min … max):  1.099 μs …  1.382 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.173 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   2.162 μs ± 22.982 μs  ┊ GC (mean ± σ):  1.67% ± 0.99%

  ▇█▅▅▂▂▁▂▁                                                  ▂
  █████████▇█▇▇▆▇▇▆▆▅▆▄▆▅▅▆▅▆▅▅▅▄▅▅▅▁▅▃▁▁▄▄▃▄▃▃▃▁▅▁▃▃▃▅▄▃▁▃▃ █
  1.1 μs       Histogram: log(frequency) by time     4.43 μs <

 Memory estimate: 752 bytes, allocs estimate: 10.

Tuple:

beta = (a = 1.0, b = 0.2, t = (0.0, 0.2, 1.0))
@benchmark irf(GPCM, $0.0, $beta)

BenchmarkTools.Trial: 10000 samples with 989 evaluations.
 Range (min … max):   79.652 ns …  13.402 μs  ┊ GC (min … max):  0.00% … 32.17%
 Time  (median):      88.914 ns               ┊ GC (median):     0.00%
 Time  (mean ± σ):   109.176 ns ± 244.511 ns  ┊ GC (mean ± σ):  11.25% ±  5.87%

  ▁█▆▅▁                                                          
  ██████▆▅▅▄▃▄▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▂▂▂▁▂▂▂▂ ▃
  79.7 ns          Histogram: frequency by time          227 ns <

 Memory estimate: 176 bytes, allocs estimate: 2.

SVector:

beta = (a = 1.0, b = 0.2, t = @SVector [0.0, 0.2, 1.0])
@benchmark irf(GPCM, $0.0, $beta)

BenchmarkTools.Trial: 10000 samples with 980 evaluations.
 Range (min … max):   78.514 ns …   7.058 μs  ┊ GC (min … max):  0.00% … 0.00%
 Time  (median):      87.862 ns               ┊ GC (median):     0.00%
 Time  (mean ± σ):   108.299 ns ± 218.631 ns  ┊ GC (mean ± σ):  11.37% ± 5.85%

   █▇▅▁▁                                                         
  ▅█████▇▆▅▄▄▄▃▃▃▃▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▂▂▂▂▂ ▃
  78.5 ns          Histogram: frequency by time          210 ns <

 Memory estimate: 176 bytes, allocs estimate: 2.
p-gw commented 4 months ago

On 0.1.5:

Vector:

julia> beta = (a = 1.0, b = 0.2, t = [0.0, 0.2, 1.0])
(a = 1.0, b = 0.2, t = [0.0, 0.2, 1.0])

julia> @benchmark irf(GPCM, $0.0, $beta)
BenchmarkTools.Trial: 10000 samples with 130 evaluations.
 Range (min … max):  729.985 ns …  44.825 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     758.169 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   835.101 ns ± 828.777 ns  ┊ GC (mean ± σ):  2.49% ± 3.01%

  ▅█▆▅▄▄▃▂▂▁▁▁                                                  ▂
  ██████████████▇███▇▇▇▆▇▆▆▇▆▆▆▅▆▆▅▆▆▆▆▆▆▆▆▆▆▅▄▅▄▃▄▄▁▅▅▄▁▅▄▄▃▄▄ █
  730 ns        Histogram: log(frequency) by time       1.76 μs <

 Memory estimate: 352 bytes, allocs estimate: 12.

Tuple:

julia> beta = (a = 1.0, b = 0.2, t = (0.0, 0.2, 1.0))
(a = 1.0, b = 0.2, t = (0.0, 0.2, 1.0))

julia> @benchmark irf(GPCM, $0.0, $beta)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  75.112 ns …   7.717 μs  ┊ GC (min … max): 0.00% … 98.78%
 Time  (median):     77.678 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   87.633 ns ± 104.845 ns  ┊ GC (mean ± σ):  2.80% ±  4.08%

  █▇▅▃▂▁▁▁▁                                                    ▂
  ████████████▇▇▇▇▇▇▇▇▇▆▆▆▅▆▆▅▅▆▄▅▅▅▅▅▅▅▅▄▄▃▅▄▄▄▅▅▁▄▄▃▁▄▃▃▅▅▄▄ █
  75.1 ns       Histogram: log(frequency) by time       238 ns <

 Memory estimate: 96 bytes, allocs estimate: 1.