SciSharp / NumSharp

High Performance Computation for N-D Tensors in .NET, similar API to NumPy.
https://github.com/SciSharp
Apache License 2.0
1.37k stars 192 forks source link

Multidimensional Array in constructor #367

Closed domtheluck closed 4 years ago

domtheluck commented 4 years ago

Hi,

I wonder if it's could be possible to initialize a NDArray with a multi dimensional array without using FromMultiDimArray method and by avoiding memory copy?

I thinking about maybe using a unmanaged pointer or something like that.

Right now, this is blocking us to adopt NumSharp because the initialization cost is too high.

Thanks

Nucs commented 4 years ago

NDArray.FromMultiDimArray<T>(Array dotNetArray) uses np.array(arr) internally but it seems to call the wrong overload.

Replace the NDArray.FromMultiDimArray<T>(Array dotNetArray) call with np.array(arr, copy: false) (make sure not to pass an non-generic System.Array so the right overload will be called).

Nucs commented 4 years ago

It was fixed in the mentioned commit above and will be available in the next release. For now, let me know if using np.array has fixed the problem for you.

domtheluck commented 4 years ago

Thanks for the quick answer :)! I will try it tomorrow and let you know if it's working.

domtheluck commented 4 years ago

np.array is actually slower than NDArray.FromMultiDimArray: Stopwatch stopWatch1 = new Stopwatch(); Stopwatch stopWatch2 = new Stopwatch();

stopWatch1.Start(); NDArray data1 = np.array( inputData.Data as Array, false ); stopWatch1.Stop();

ElapsedMilliseconds: 81

stopWatch2.Start(); NDArray data2 = NDArray.FromMultiDimArray<Double>( inputData.Data as Array ); stopWatch2.Stop();

ElapsedMilliseconds: 57

inputData.Data: double[1, 1, 29600, 16]

Nucs commented 4 years ago

(make sure >>>> not <<<< to pass an non-generic System.Array so the right overload will be called).

You should not use System.Array. This sends you to the following overload:

public static NDArray array(NDArray nd, bool copy = false) => copy ? new NDArray(nd.Storage.Clone()) : new NDArray(nd.Storage);

Passing System.Array calls implicit cast to NDArray.

You should use one of these overloads: image

I guarantee it will run in less than 5ms. I dare to say <1ms.

domtheluck commented 4 years ago

Sorry my bad ... :)!