search

ruby-numo / numo-narray

Ruby/Numo::NArray - New NArray class library
http://ruby-numo.github.io/narray/
BSD 3-Clause "New" or "Revised" License
415 stars 41 forks source link

Loading Binary Data #140

Closed ankane closed 5 years ago

ankane commented 5 years ago

Hi, I'm working on loading Numpy .npy and .npz files into an NArray (which maybe we can include in this project at some point). The data can be read into Ruby with:

arr = byte_str.unpack("C*")

And later converted with:

Numo::UInt8[*arr]

However, it'd be great if there was a way to read directly into an NArray for efficiency (also, the current approach causes a stack level too deep error with large arrays - see #139 for more details).

Here's the code and test data.

mrkn commented 5 years ago

@ankane I guess Numo::NArray.from_string is appropriate for your purpose if you can detect the value type of deserialized arrays.

masa16 commented 5 years ago

Yes, Numo::UInt8.from_string(byte_str) is faster for conversion from binary string.

kojix2 commented 5 years ago

https://rdoc.info/gems/numo-narray/Numo%2FNArray%2Efrom_binary

ankane commented 5 years ago

Not sure how I missed that (facepalm). I appreciate the help.

For future readers, you need to call it on a specific type, as calling it on Numo::NArray raises uninitialized constant Numo::NArray::ELEMENT_BYTE_SIZE.

ankane commented 5 years ago

Here's the gem, just fyi: https://github.com/ankane/npy