It reduces memory usage when NArray just uses data as read only data
because store_binary doesn't copy the passed binary data until NArray
writes the passed binary data.
Benchamrk:
require "numo/narray"
GC.disable
shape = [100, 100]
array = Numo::Int32.new(*shape).seq
data = array.to_binary
frozen_data = array.to_binary.freeze
marshaled_data = Marshal.dump(array)
def report_memory_usage(label, previous)
usage = File.readlines("/proc/self/status").grep(/^VmRSS/)[0][/^VmRSS:\s*(\d+)/, 1].to_i
puts "#{label}: #{usage - previous} kB"
usage
end
usage = report_memory_usage("initial", 0)
10000.times do
Numo::Int32.new(*shape).seq
end
usage = report_memory_usage("seq", usage)
10000.times do
Numo::Int32.from_binary(data, shape)
end
usage = report_memory_usage("from not frozen data", usage)
10000.times do
Numo::Int32.from_binary(frozen_data, shape)
end
usage = report_memory_usage("from frozen data", usage)
10000.times do
Numo::Int32.from_binary(frozen_data, shape)[0, 0] = 1
end
usage = report_memory_usage("from frozen data with write", usage)
10000.times do
Marshal.load(marshaled_data)
end
usage = report_memory_usage("marshal", usage)
With this change:
initial: 16488 kB
seq: 393360 kB
from not frozen data: 392040 kB
from frozen data: 1848 kB
from frozen data with write: 392660 kB
marshal: 396528 kB
Without this change:
initial: 16276 kB
seq: 393360 kB
from not frozen data: 391776 kB
from frozen data: 392040 kB
from frozen data with write: 392040 kB
marshal: 786456 kB
See "fron frozen data" entry. It reduces memory usage from 392040 kB
to 1848 kB (0.4%).
See also "marshal" entry. It reduces memory usage from 786456 kB
to 396528 kB (50%).
It reduces memory usage when NArray just uses data as read only data because store_binary doesn't copy the passed binary data until NArray writes the passed binary data.
Benchamrk:
With this change:
Without this change:
See "fron frozen data" entry. It reduces memory usage from 392040 kB to 1848 kB (0.4%).
See also "marshal" entry. It reduces memory usage from 786456 kB to 396528 kB (50%).