mkitti / ArrayAllocators.jl

Allocate arrays with malloc, calloc, or on NUMA nodes
MIT License
53 stars 1 forks source link

It seems no improvement when the dimension is greater than 2 #7

Closed SeanLee97 closed 2 years ago

SeanLee97 commented 2 years ago

It seems that ArrayAllocators does not perform well for multi-dimension Arrays. Here enclosed my experimental results:

julia> import ArrayAllocators

julia> ArrayAllocators.zeros(Int, 28, 6, 6, 28239);

julia> @time ArrayAllocators.zeros(Int, 28, 6, 6, 28239);
  0.132088 seconds (3 allocations: 217.170 MiB)

julia> Base.zeros(Int, 28, 6, 6, 28239);

julia> @time Base.zeros(Int, 28, 6, 6, 28239);
  0.128121 seconds (3 allocations: 217.170 MiB)

julia> ArrayAllocators.Array{Int, 4}(undef, (28, 6, 6, 28239));

julia> @time ArrayAllocators.Array{Int, 4}(undef, (28, 6, 6, 28239));
  0.147250 seconds (2 allocations: 217.170 MiB, 99.97% gc time)

julia> Base.Array{Int, 4}(undef, (28, 6, 6, 28239));

julia> @time Base.Array{Int, 4}(undef, (28, 6, 6, 28239));
  0.000027 seconds (2 allocations: 217.170 MiB)

It shows that: 1) Base.zeros is more efficient than ArrayAllocators.zeros 2) Base.Array is far more efficient than ArrayAllocators.Array

environments

Julia version: 1.7.2 ArrayAllocators version: v0.2.0

SeanLee97 commented 2 years ago

It seems that ArrayAllocators does not perform well for multi-dimension Arrays. Here enclosed my experimental results:

julia> import ArrayAllocators

julia> ArrayAllocators.zeros(Int, 28, 6, 6, 28239);

julia> @time ArrayAllocators.zeros(Int, 28, 6, 6, 28239);
  0.132088 seconds (3 allocations: 217.170 MiB)

julia> Base.zeros(Int, 28, 6, 6, 28239);

julia> @time Base.zeros(Int, 28, 6, 6, 28239);
  0.128121 seconds (3 allocations: 217.170 MiB)

julia> ArrayAllocators.Array{Int, 4}(undef, (28, 6, 6, 28239));

julia> @time ArrayAllocators.Array{Int, 4}(undef, (28, 6, 6, 28239));
  0.147250 seconds (2 allocations: 217.170 MiB, 99.97% gc time)

julia> Base.Array{Int, 4}(undef, (28, 6, 6, 28239));

julia> @time Base.Array{Int, 4}(undef, (28, 6, 6, 28239));
  0.000027 seconds (2 allocations: 217.170 MiB)

It shows that:

  1. Base.zeros is more efficient than ArrayAllocators.zeros
  2. Base.Array is far more efficient than ArrayAllocators.Array

environments

Julia version: 1.7.2 ArrayAllocators version: v0.2.0

Oh sorry, I found the ArrayAllocators.Array includes the GC time. I rerun the experiment and see that the Base.Array is slightly better than ArrayAllocators'

julia> @time ArrayAllocators.Array{Int, 4}(calloc, (28, 6, 6, 28239));
  0.000019 seconds (2 allocations: 217.170 MiB)

julia> @time Base.Array{Int, 4}(undef, (28, 6, 6, 28239));
  0.000014 seconds (2 allocations: 217.170 MiB)
mkitti commented 2 years ago

Perhaps there is some confusion. ArrayAllocators.jl does not have its own zeros or Array methods. They are the same ones from Base.

What is new is that you can do Base.Array{Int}(ArrayAllocators.calloc, 1024).

mkitti commented 2 years ago

To elaborate further, ArrayAllocators.Array and Base.Array both refer to the same Core.Array. ArrayAllocators.zeros is identical to Base.zeros.

julia> using ArrayAllocators

julia> @which ArrayAllocators.Array
Core

julia> @which Base.Array
Core

julia> @which ArrayAllocators.zeros
Base

julia> @which Base.zeros
Base

julia> Base.Array === ArrayAllocators.Array
true

julia> Base.zeros === ArrayAllocators.zeros
true
mkitti commented 2 years ago

The advantage that this package provides is that you can use calloc. Compare that to using zeros.

julia> @time Z = Base.zeros(Int, 28, 6, 6, 28239);
  0.229623 seconds (3 allocations: 217.170 MiB, 48.00% gc time)

julia> @time C = Base.Array{Int}(calloc, 28, 6, 6, 28239);
  0.031148 seconds (64.67 k allocations: 220.960 MiB, 99.90% compilation time)

julia> isequal(Z, C)
true

Note that using undef will not initialize your Array with 0.

mkitti commented 2 years ago

On a better computer, I get this after a second execution:

julia> @time Z = Base.zeros(Int, 28, 6, 6, 28239);
  0.137507 seconds (3 allocations: 217.170 MiB, 4.23% gc time)

julia> @time C = Base.Array{Int}(calloc, 28, 6, 6, 28239);
  0.000021 seconds (4 allocations: 217.170 MiB)

julia> isequal(Z,C)
true
SeanLee97 commented 2 years ago

Many thanks for your detailed explanation :)