SciSharp / NumSharp

High Performance Computation for N-D Tensors in .NET, similar API to NumPy.
https://github.com/SciSharp
Apache License 2.0
1.36k stars 189 forks source link

np.load cannot load Python Objects #404

Closed tk4218 closed 4 years ago

tk4218 commented 4 years ago

I have a .npy file that I am trying to load with this header: {'descr': '|O', 'fortran_order': False, 'shape': (), }

In np.load(), parseReader() will throw an exception when it calls the GetType() function on this line: bytes = Int32.Parse(dtype.Substring(2));

This is because '|O' is only two characters long and doesn't have the byte count character on the end. Also, it doesn't look like the "O" dtype is supported throughout Numsharp - understandably due to there not being an equivalent to a Python Object in .NET.

Are there any plans to do something to support loading .npy files with Python Objects, and potentially have an object that is the equivalent of a Python Object?

Nucs commented 4 years ago

Unfortunately it is not possible to load or create a NDArray as Data type Objects. NumSharp only supports native built-in dtypes like int32, float32 (float), float64 (double). I suggest you to try Numpy.NET.

tk4218 commented 4 years ago

FWIW, it seems like you can convert pickled Python objects to JSON using Python, then load the JSON instead into the program using Numsharp.

In my case, I'm trying to avoid the use of Python altogether on the machine I'm running my Numsharp program, however I have no problem doing the JSON conversion on another machine using Python or Numpy.NET. One downside to using JSON over .npy files is that the file size is roughly 5-6 time larger, however, this solution seems to work.