not implemented for `f64`

[nilgoyette]

I'm trying to load a npy file into a Array<f64, 2> (from rust-ndarray) and I'm somewhat surprised that it doesn't work. My code is:

let mut buf = vec![];
let f = File::open("examples/faces.npy").unwrap();
let mut reader = BufReader::new(f);
reader.read_to_end(&mut buf).unwrap();
let faces = Array::from_iter(
    NpyData::from_bytes(&buf).unwrap().into_iter()
).into_shape((1444, 3)).unwrap();

and this error is repeated 3 times:

the trait bound `f64: npy::NpyRecord` is not satisfied

Should it work? Or are we forced to call to_vec before?

[nilgoyette]

In fact, adding .to_vec() gives the same error.

I don't understand why I can loop on it and push each element in a vector, but not use into_iter().

[nilgoyette]

I think I understand! NpyRecord is not defined for any type, we define it with #[derive(NpyRecord, Debug)] for the type(s) we want to work with.

Would it be possible to define it by default for all POD? i8, u8, i16, u16, etc. I'm pretty sure most numpy data are PODs, not structs.

[potocpav]

Yes, you understand it correctly. It is definitely possible to define NpyRecord for the primitive types. It may require some bigger code changes, since we probably want to serialize primitive type arrays to a simple Numpy array (not a structured array). I will do it as soon as I find some time. Meanwhile, you can create a custom struct and copy the data over.

[nilgoyette]

No stress, take your time. I'll probably roll my own reader because I realized that I also need 'shape': (1444, 3) and you don't support shape.len() > 1. This makes 2 big issues and you probably don't have the time for that right now!

[potocpav]

Yes, 2d arrays aren't implemented either. Share your code if possible :)

[pkgw]

I'm also interested in this functionality. I have existing files that are stored as 1D f64 vectors, but I don't think there's a way for me to read them in right now. As far as I can tell, a custom struct that consists of a single scalar f64 isn't considered compatible with unstructured f64 data.

[milibopp]

Any progress on the shape issue? I am working around that at the moment and would like to add support for shapes to npy, if nobody else is actively doing that right now.

[potocpav]

No, no progress yet :-/ Sorry for the late reply - vacation.

[milibopp]

For the record, I've picked up working on this now. It definitely requires a breaking change to both npy and npy-derive.

[milibopp]

The current state is on this branch. So far I have only introduced a new wrapper type for the compound dtype of a record.

[potocpav]

Thanks to aepsil0n, this issue is now solved: arrays of primitive types can be (de)serialized.