What is issubdtype at numpy.net

[ChengYen-Tang]

Hi, I can't found issubdtype at numpy.net. Is there any alternative solution https://numpy.org/doc/stable/reference/generated/numpy.issubdtype.html

[KevinBaselinesw]

Good catch. Some properties like issubdtype already existed but were not public. I made them public and added a few more. Get the latest version 0.9.80 to get these features.

Below is a unit test that shows all of the properties that should allow numpydotnet to have the same functionality as issubdtype.

    [TestMethod]
        public void test_array_dtype_checks()
        {
            var a = np.array(new int[] { 1, 2 });
            Assert.AreEqual(NPY_TYPES.NPY_INT32, a.Dtype.TypeNum);
            Assert.AreEqual(true,a.IsInteger);
            Assert.AreEqual(true, a.IsNumber);
            Assert.AreEqual(true, a.IsSignedInteger);
            Assert.AreEqual(false, a.IsUnsignedInteger);
            Assert.AreEqual(false, a.IsComplex);
            Assert.AreEqual(false, a.IsFloatingPoint);
            Assert.AreEqual(false, a.IsInexact);
            Assert.AreEqual(false, a.IsDecimal);

            a = np.array(new double[] { 1, 2 });
            Assert.AreEqual(NPY_TYPES.NPY_DOUBLE, a.Dtype.TypeNum);
            Assert.AreEqual(false, a.IsInteger);
            Assert.AreEqual(true, a.IsNumber);
            Assert.AreEqual(false, a.IsSignedInteger);
            Assert.AreEqual(false, a.IsUnsignedInteger);
            Assert.AreEqual(false, a.IsComplex);
            Assert.AreEqual(true, a.IsFloatingPoint);
            Assert.AreEqual(true, a.IsInexact);
            Assert.AreEqual(false, a.IsDecimal);

            a = np.array(new Complex[] { 1, 2 });
            Assert.AreEqual(NPY_TYPES.NPY_COMPLEX, a.Dtype.TypeNum);
            Assert.AreEqual(false, a.IsInteger);
            Assert.AreEqual(true, a.IsNumber);
            Assert.AreEqual(false, a.IsSignedInteger);
            Assert.AreEqual(false, a.IsUnsignedInteger);
            Assert.AreEqual(true, a.IsComplex);
            Assert.AreEqual(true, a.IsFloatingPoint);
            Assert.AreEqual(true, a.IsInexact);
            Assert.AreEqual(false, a.IsDecimal);

            a = np.array(new Decimal[] { 1, 2 });
            Assert.AreEqual(NPY_TYPES.NPY_DECIMAL, a.Dtype.TypeNum);
            Assert.AreEqual(false, a.IsInteger);
            Assert.AreEqual(true, a.IsNumber);
            Assert.AreEqual(false, a.IsSignedInteger);
            Assert.AreEqual(false, a.IsUnsignedInteger);
            Assert.AreEqual(false, a.IsComplex);
            Assert.AreEqual(true, a.IsFloatingPoint);
            Assert.AreEqual(true, a.IsInexact);
            Assert.AreEqual(true, a.IsDecimal);

            a = np.array(new BigInteger[] { -1, 2 });
            Assert.AreEqual(NPY_TYPES.NPY_BIGINT, a.Dtype.TypeNum);
            Assert.AreEqual(true, a.IsInteger);
            Assert.AreEqual(true, a.IsNumber);
            Assert.AreEqual(true, a.IsSignedInteger);
            Assert.AreEqual(false, a.IsUnsignedInteger);
            Assert.AreEqual(false, a.IsComplex);
            Assert.AreEqual(false, a.IsFloatingPoint);
            Assert.AreEqual(false, a.IsInexact);
            Assert.AreEqual(false, a.IsDecimal);

            a = np.array(new bool[] { true, false });
            Assert.AreEqual(NPY_TYPES.NPY_BOOL, a.Dtype.TypeNum);
            Assert.AreEqual(true, a.IsBool);
            Assert.AreEqual(false, a.IsInteger);
            Assert.AreEqual(false, a.IsNumber);
        }

[ChengYen-Tang]

I'm thinking about some questions numpy almost needs type checking because python is a feature of weak typing language, but this violates the open-closed principle, which leads to several problems.

1. I want to develop a reinforcement learning environment framework like openai/gym, but the developers who use this framework don't know the content type when they get ndarray, because ndarray returns the object type.
2. For objects that do not know a clear type, we need to use if...else... for checking and conversion, which increases maintenance costs and violates OCP. At the same time, in reinforcement learning, each training requires millions of steps to tens of millions of iterations, which will increase the huge time complexity.

Should ndarray use generics to mitigate these side effects?

[KevinBaselinesw]

I don't want to rewrite numpydotnet to use generics. I studied that at one time and concluded there was not all that much to gain. I especially wanted to keep the API as close the python as possible and making it generic would violate that.

What could be done is to create a generic wrapper around the ndarray class. It might look like this below. Feel free to experiment with this model. If you can come up with a helpful usage, I can consider adding it to the numpydotnet library.

   public class ndarray<T>
    {
        public T[] data
        {
            get
            {
                return arr.ToArray<T>();
            }
        }

        private ndarray arr;
        public static ndarray<T> array(T[] data)
        {
            ndarray arr = np.array(data);
            ndarray<T> tarr = new ndarray<T>();
            tarr.arr = arr;
            return tarr;
        }

        public T[] Sum()
        {
            var x = arr.Sum();
            return x.ToArray<T>();
        }
    }

    [TestClass]
    public class GenericArrayFunctions : TestBaseClass
    {
        [TestMethod]
        public void test_generic_array_1()
        {
            var ta = ndarray<Int32>.array(new Int32[] { 1, 2, 3 });

            var data = ta.data;
            var sum = ta.Sum();

        }

    }

[KevinBaselinesw]

Here is a slightly improved generics class

   public class ndarray<T>
    {
        public T[] data
        {
            get
            {
                return arr.ToArray<T>();
            }
        }

        private ndarray arr;

        public ndarray()
        {

        }
        public ndarray(ndarray arr)
        {
            this.arr = arr;
        }

        public static ndarray<T> array(T[] data)
        {
            ndarray arr = np.array(data);
            ndarray<T> tarr = new ndarray<T>(arr);
            return tarr;
        }
        public static ndarray<T> array(T[,] data)
        {
            ndarray arr = np.array(data);
            ndarray<T> tarr = new ndarray<T>(arr);
            return tarr;
        }

        public ndarray<T> Sum(int? axis = null)
        {
            var x = arr.Sum(axis);

            ndarray<T> ret = new ndarray<T>(x);
            return ret;
        }
    }

    [TestClass]
    public class GenericArrayFunctions
    {
        [TestMethod]
        public void test_generic_array_1()
        {
            var ta = ndarray<Int32>.array(new Int32[] { 1, 2, 3 });

            var data = ta.data;
            var sum = ta.Sum();
        }

        [TestMethod]
        public void test_generic_array_2()
        {
            var ta = ndarray<Int32>.array(new Int32[,] { { 1, 2, 3 }, { 1, 2, 3 } });

            var data = ta.data;
            var sum0 = ta.Sum(0);
            var sum1 = ta.Sum(1);
        }
    }

[ChengYen-Tang]

Umm... this approach seems to have performance issues. The classic environment for reinforcement learning is the atari game, which usually uses screen images as observational input. So the size of the ndarray will be greater than 1MB.

Using ToArray runtime will copy data. Large data copy is time-consuming, and it will put pressure on the GC and increase the time and number of program pauses. This performance optimization problem may be the next milestone for this package.

Several key points of performance optimization

1. Reduce data copying
2. Use memory pool or array pool
3. Using Span

[KevinBaselinesw]

ToArray should only copy the data if the ndarray object IsASlice. That means that someone has done something like:

a= a["1",50","2"];

If the array is not sliced, it will just return a reference to the array.

If you want, write some sample code with a problem you are trying to solve and I will work with you to come up with a solution.

[ChengYen-Tang]

sorry for my fault I thought this ToArray by linq. But I think you can consider using arraypool as the bottom layer of ndarray to reuse the allocated memory, this method has obvious performance improvement in video streaming.

Is this shape correct?

[KevinBaselinesw]

      [TestMethod]
        public void test_generic_array_3()
        {
            ndarray objectarray = np.array(new int[][] { new int[] { 1, 2, 3 }, new int[] { 1, 2, 3 } });
            print(objectarray);

            var multidim_intarray = np.array(new Int32[,] { { 1, 2, 3 }, { 1, 2, 3 } });
            print(multidim_intarray);

        }

object array is initialized as an array of arrays. We don't know what to do with that so we default that to an object array. I don't think I can apply any numpy operations to this so it is likely not valid.

shape=(2,), OBJECT { System.Int32[], System.Int32[] }

multidim_intarray is maybe what you are trying to do. shape=(2, 3), INT32 { { 1, 2, 3 }, { 1, 2, 3 } }

[ChengYen-Tang]

Umm... If you call ToArray() from List< List < int > >, it will be int[][] instead of int[,]

In python, if the array is rectangular, numpy can automatically convert it, if it is irregular, it will be an object array.

[ChengYen-Tang]

I found strange problem... I upgraded your unit test project to .net6 it doesn't compile

From a code point of view, there shouldn't be any of these places throwing errors.

Step1: Content that replaces NumpyDotNetTests.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.msbuild" Version="3.1.2">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="MathNet.Numerics" Version="5.0.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.10" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.10" />
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="OpenCover" Version="4.7.1221" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\NumpyDotNet\NumpyDotNet.csproj" />
  </ItemGroup>
</Project>

Step2: Delete NumpyDotNet\UnitTests\NumpyDotNetTests\Properties\AssemblyInfo

[ChengYen-Tang]

I have a new problem.

I can do this in python.

>>> a = np.array([1, 2, 3])
>>> -np.inf < a
array([ True,  True,  True])

But how can I do this in C#?

ndarray a = np.empty(new shape(new int[] { 1, 2, 3 }), np.Int32);
-np.inf < a 🤔

[KevinBaselinesw]

Use the np.less() function as below.

      [TestMethod]
        public void test_ChengYenTang_1()
        {
            var a = np.array(new int[] { 1, 2, 3 });

            var b = np.less(double.NegativeInfinity, a);
            AssertArray(b, new bool[] { true, true, true });
            print(b);

            var c = a > double.NegativeInfinity;
            AssertArray(c, new bool[] { true, true, true });
            print(c);
        }``

[KevinBaselinesw]

I just added some additional overrides that will allow this syntax. It will be in the next release.

var c = double.NegativeInfinity < a;

[ChengYen-Tang]

https://numpy.org/doc/stable/reference/random/generated/numpy.random.uniform.html

In the python version of numpy.random.uniform, high and low can support arrays. But the C# version seems to only support numeric values.

[KevinBaselinesw]

Do you have a sample python call that passes an array to np.random.uniform? If so, I will try to make it work for you.

[ChengYen-Tang]

Is this ok?

>>> a = np.array([9.0, 8.0, 7.0, 1.0])
>>> low = np.array([9.0, 8.0, 7.0, 1.0])
>>> high = np.array([30.0, 22.0, 10.0, 3.0])
>>> shape = (4,)
>>> np.random.uniform(low, high, shape)
array([19.95860237,  8.7686677 ,  7.46338379,  2.57969312])

I would like to add one thing, the official provides the default parameters for the following two methods, maybe this project can also add default parameters. https://numpy.org/doc/stable/reference/random/generated/numpy.random.normal.html https://numpy.org/doc/stable/reference/random/generated/numpy.random.exponential.html

[KevinBaselinesw]

Update to the newest package 0.9.80.2 to get these updated APIs.

[ChengYen-Tang]

Thank you for your active support.

[ChengYen-Tang]

in the new version I have an error in my original code.

  Low = np.array(new double[2, 3] { {9, 8, 7 }, { 2, double.NegativeInfinity, 1 } });
  High = np.array(new double[2, 3] { { 30, 22, 10 }, { double.PositiveInfinity, 5, 3 } });
  Console.WriteLine(Low < High);    // CS0034 Operator '<' is ambiguous on operands of type 'ndarray' and 'ndarray'

[KevinBaselinesw]

Get the latest 0.9.80.3 for this fix.

     [TestMethod]
        public void test_ChengYenTang_2()
        {
            var low = np.array(new double[2, 3] { { 30, 8, 7 }, { 2, double.NegativeInfinity, 3 } });
            var high = np.array(new double[2, 3] { { 30, 22, 10 }, { double.PositiveInfinity, 5, 3 } });

            var a = low < high;
            AssertArray(a, new bool[,] { { false, true, true }, { true, true, false } });
            print(a);

            var b = low > high;
            AssertArray(b, new bool[,] { { false, false, false }, { false, false, false } });
            print(b);

            var c = low <= high;
            AssertArray(c, new bool[,] { { true, true, true }, { true, true, true } });
            print(c);

            var d = low >= high;
            AssertArray(d, new bool[,] { { true, false, false }, { false, false, true } });
            print(d);

        }

[ChengYen-Tang]

Can numpy.net support shape+shape? like python code:

(2,) + (3,)
>>> (2, 3)

[KevinBaselinesw]

I think this is more a python vs C# issue than a numpy issue. In your sample above, I think python is adding two arrays together. Here is what I propose as a solution. Let me know if it works for you.

Python unit test
def test_ChengYenTang_3(self):

        a = np.arange(0,32);
        b = np.reshape(a, (2,)+(16,))
        print(b.shape)

        c = np.reshape(a, (2,2)+(8,))
        print(c.shape)

        d = np.reshape(a, (2,2)+(2,4))
        print(d.shape)

c# unit test [TestMethod] public void test_ChengYenTang_3() { var a = np.arange(0,32); var b = np.reshape(a, new shape(2) + new shape(16)); print(b.shape);

        var c = np.reshape(a, new shape(2,2) + new shape(8));
        print(c.shape);

        var d = np.reshape(a, new shape(2, 2) + new shape(2,4));
        print(d.shape);

    }

proposed addition to NumpyDotNet to make it work similarly to Python.

     /// <summary>
        /// add two shapes together
        /// </summary>
        /// <param name="a"></param>
        /// <param name="b"></param>
        /// <returns></returns>
        public static shape operator +(shape a, shape b)
        {
            npy_intp[] newdims = new npy_intp[a.iDims.Length + b.iDims.Length];

            Array.Copy(a.iDims, 0, newdims, 0, a.iDims.Length);
            Array.Copy(b.iDims, 0, newdims, a.iDims.Length, b.iDims.Length);

            return new shape(newdims);
        }

[KevinBaselinesw]

I have extended the shape API to accept multiple arrays as inputs. These might be helpful.

        [TestMethod]
        public void test_ChengYenTang_3()
        {
            var a = np.arange(0,32);
            var b = np.reshape(a, new shape(2) + new shape(16));
            print(b.shape);

            var c = np.reshape(a, new shape(2,2) + new shape(8));
            print(c.shape);

            var d = np.reshape(a, new shape(2, 2) + new shape(2,4));
            print(d.shape);

            var e = np.reshape(a, new shape(new int[] { 2, 2 }, new int[] { 2, 4 }));
            print(e.shape);

            var f = np.reshape(a, new shape(new long[] { 2, 2 }, new long[] { 2, 4 }));
            print(f.shape);

            var g = np.reshape(a, new shape(new int[] { 2, 2 }, new int[] { 2, 4 }, new int[] { 1, 1 }));
            print(g.shape);

            var h = np.reshape(a, new shape(new long[] { 2, 2 }, new long[] { 2, 4 }, new long[] { 1, 1 }));
            print(h.shape);

        }

[ChengYen-Tang]

Thanks, this helped me a lot

[KevinBaselinesw]

0.9.80.4 has the official updates for this.

[ChengYen-Tang]

Hi, Can you help me with this problem?

Python:

>>> low = np.array([[9, 8, 7], [2, -np.inf, 1]])
>>> low
array([[ 9.,  8.,  7.],
       [ 2., -inf,  1.]])
>>> stack_low = np.repeat(low, 3, axis=0)
>>> stack_low
array([[  9.,   8.,   7.],
       [  9.,   8.,   7.],
       [  9.,   8.,   7.],
       [  2., -inf,   1.],
       [  2., -inf,   1.],
       [  2., -inf,   1.]])
>>> observation = np.array([[[9, 8, 7], [2, -np.inf, 1]], [[30, 22, 10], [np.inf, 5, 3]]])
>>> observation
array([[[  9.,   8.,   7.],
        [  2., -inf,   1.]],

       [[ 30.,  22.,  10.],
        [ inf,   5.,   3.]]])
>>> stackedobs = np.zeros((2,) + stack_low.shape);
>>> stackedobs
array([[[0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.]],

       [[0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.],
        [0., 0., 0.]]])
>>> stackedobs[:, -observation.shape[1] :, ...] = observation
>>> stackedobs
array([[[  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [  9.,   8.,   7.],
        [  2., -inf,   1.]],

       [[  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [  0.,   0.,   0.],
        [ 30.,  22.,  10.],
        [ inf,   5.,   3.]]])

C# :

>>> ndarray low = np.array(new double[2, 3] { { 9, 8, 7 }, { 2, double.NegativeInfinity, 1 } });
>>> Console.WriteLine(low);
shape=(2, 3), DOUBLE
{ { 9.0, 8.0, 7.0 },
  { 2.0, -∞.0, 1.0 } }
>>> ndarray stack_low = np.repeat(low, 3, axis: 0);
>>> Console.WriteLine(stack_low);
shape=(6, 3), DOUBLE
{ { 9.0, 8.0, 7.0 },
  { 9.0, 8.0, 7.0 },
  { 9.0, 8.0, 7.0 },
  { 2.0, -∞.0, 1.0 },
  { 2.0, -∞.0, 1.0 },
  { 2.0, -∞.0, 1.0 } }
>>> ndarray observation = np.array(new double[2, 2, 3] { { { 9, 8, 7 }, { 2, double.NegativeInfinity, 1 } }, { { 30, 22, 10 }, { double.PositiveInfinity, 5, 3 } } });
>>> Console.WriteLine(observation);
shape=(2, 2, 3), DOUBLE
{ { { 9.0, 8.0, 7.0 },
    { 2.0, -∞.0, 1.0 } },
  { { 30.0, 22.0, 10.0 },
    { ∞.0, 5.0, 3.0 } } }
>>> ndarray stackedobs = np.zeros(new shape(2) + stack_low.shape);
>>> Console.WriteLine(stackedobs);
shape=(2, 6, 3), DOUBLE
{ { { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 } },
  { { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 0.0 } } }
>>> stackedobs[$":, -{observation.shape[1]} :, ..."] = observation;
System.Exception: '(NpyExc_ValueError) NpyArray_Broadcast: shape mismatch: objects cannot be broadcast to a single shape'

[KevinBaselinesw]

stackedobs[":", $"-{observation.shape[1]}:", "..."] = observation;

Each index item needs to be a separate string like this.

    [TestMethod]
        public void test_ChengYenTang_4()
        {
            ndarray low = np.array(new double[2, 3] { { 9, 8, 7 }, { 2, double.NegativeInfinity, 1 } });
            print(low);

            ndarray stack_low = np.repeat(low, 3, axis: 0);
            print(stack_low);

            ndarray observation = np.array(new double[2, 2, 3] { { { 9, 8, 7 }, { 2, double.NegativeInfinity, 1 } }, { { 30, 22, 10 }, { double.PositiveInfinity, 5, 3 } } });
            print(observation);

            ndarray stackedobs = np.zeros(new shape(2) + stack_low.shape);
            print(stackedobs);

            stackedobs[":", $"-{observation.shape[1]}:", "..."] = observation;
            print(stackedobs);

        }

[ChengYen-Tang]

Thank you for your help

[ChengYen-Tang]

I have a question why you use object type instead of dynamic type. With dynamic types we don't need to worry about type conversions

[KevinBaselinesw]

1) dynamic keyword is very, very slow compared to using C# native types. 2) some .NET implementations do not support dynamic keyword. For example, a user of NumpyDotNet was building a HoloLens application. It was throwing an exception if it hit dynamic. I worked with them to reduce the places where dynamic could be hit.
3) Internally, NumpyDotNet does use dynamic in a few small cases when it is necessary to perform operations on different data datatypes but it is much slower. The most important thing is to use the same data types when running operations.

Is there a particular operation that you think could benefit from using dynamic rather than object? If so, can you use dynamic in the application? I think assigning an object to a dynamic variable works. If you don't mind the performance impact.

[ChengYen-Tang]

I get an error

python:

>>> a = np.array([[0], [0], [0]])
>>> a.shape
(3, 1)
>>> a.shape[-1]
1

C#

>>> var a = np.array(new int[,] { { 0 }, { 0 }, { 0 } });
>>> Console.WriteLine(a.shape);
(3, 1)
>>> Console.WriteLine(a.shape[-1]);
System.Exception: 'attempting to access shape dimension outside

[KevinBaselinesw]

Please get the latest version 0.9.80.5 that will fix this issue for you.

thank you,

Kevin

[ChengYen-Tang]

Thanks, But I got new error 😭

https://github.com/Quansight-Labs/numpy.net/blob/d264890d90cbaa050a975affdda4f7f9aef677cf/src/NumpyDotNet/NumpyDotNet/Numeric.cs#L1407

[KevinBaselinesw]

Here is the original python code that implements np.roll. Feel free to try and implement this in C#. I started it and gave up. Unfortunately I don't have time to take on a big task like that right now.

def roll(a, shift, axis=None):
    """
    Roll array elements along a given axis.

    Elements that roll beyond the last position are re-introduced at
    the first.

    Parameters
    ----------
    a : array_like
        Input array.
    shift : int or tuple of ints
        The number of places by which elements are shifted.  If a tuple,
        then `axis` must be a tuple of the same size, and each of the
        given axes is shifted by the corresponding number.  If an int
        while `axis` is a tuple of ints, then the same value is used for
        all given axes.
    axis : int or tuple of ints, optional
        Axis or axes along which elements are shifted.  By default, the
        array is flattened before shifting, after which the original
        shape is restored.

    Returns
    -------
    res : ndarray
        Output array, with the same shape as `a`.

    See Also
    --------
    rollaxis : Roll the specified axis backwards, until it lies in a
               given position.

    Notes
    -----
    .. versionadded:: 1.12.0

    Supports rolling over multiple dimensions simultaneously.

    Examples
    --------
    >>> x = np.arange(10)
    >>> np.roll(x, 2)
    array([8, 9, 0, 1, 2, 3, 4, 5, 6, 7])

    >>> x2 = np.reshape(x, (2,5))
    >>> x2
    array([[0, 1, 2, 3, 4],
           [5, 6, 7, 8, 9]])
    >>> np.roll(x2, 1)
    array([[9, 0, 1, 2, 3],
           [4, 5, 6, 7, 8]])
    >>> np.roll(x2, 1, axis=0)
    array([[5, 6, 7, 8, 9],
           [0, 1, 2, 3, 4]])
    >>> np.roll(x2, 1, axis=1)
    array([[4, 0, 1, 2, 3],
           [9, 5, 6, 7, 8]])

    """
    a = asanyarray(a)
    if axis is None:
        return roll(a.ravel(), shift, 0).reshape(a.shape)

    else:
        axis = normalize_axis_tuple(axis, a.ndim, allow_duplicate=True)
        broadcasted = broadcast(shift, axis)
        if broadcasted.ndim > 1:
            raise ValueError(
                "'shift' and 'axis' should be scalars or 1D sequences")
        shifts = {ax: 0 for ax in range(a.ndim)}
        for sh, ax in broadcasted:
            shifts[ax] += sh

        rolls = [((slice(None), slice(None)),)] * a.ndim
        for ax, offset in shifts.items():
            offset %= a.shape[ax] or 1  # If `a` is empty, nothing matters.
            if offset:
                # (original, result), (original, result)
                rolls[ax] = ((slice(None, -offset), slice(offset, None)),
                             (slice(-offset, None), slice(None, offset)))

        result = empty_like(a)
        for indices in itertools.product(*rolls):
            arr_index, res_index = zip(*indices)
            result[res_index] = a[arr_index]

        return result

[ChengYen-Tang]

I am not familiar with the architecture of this package, but I need it very much, I am developing a reinforcement learning framework similar to stable-baselines When do you have time?😫

[KevinBaselinesw]

I started working on. It is a lot of work. It will be a few days before I have it ready.

[ChengYen-Tang]

OK, thank you for your help❤️

[KevinBaselinesw]

I put a new version that should fix this np.roll issue for you.

[ChengYen-Tang]

Thank you very much, but I found new bug.

Python:

>>> a = np.array([[3,2],[3,2],[3,2]])
>>> a[0, ..., :-1]
array([3])

C#:

>>> var a = np.array(new int[,] { { 3, 2 }, { 3, 2 }, { 3, 2 } });
>>> Console.WriteLine(a[0, "...", ":-1"]);
System.IndexOutOfRangeException: 'Index was outside the bounds of the array.'

[KevinBaselinesw]

As a workaround, try this format instead. Ultimately ";-1" gets translated to Slice(null, -1, 1)

     var b = a[0, "...",new Slice(null, -1, 1)];

I will try to figure out why the string parser needs to do an array dim range check.

[ChengYen-Tang]

Is this the same problem? python:

>>> stackedobs = np.zeros((3, 12, 8, 6))
>>> stackedobs[..., -3:]
shape:   (3, 12, 8, 3)
array([[[[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]]],

       [[[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]]],

       [[[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]]]])

C#

>>>  ndarray stackedobs = np.zeros(new shape(3, 12, 8, 6));
>>> Console.WriteLine(stackedobs["...", "-3:"]);
shape=(3, 12, 8, 0), DOUBLE
{ { { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } } },
  { { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } } },
  { { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } },
    { {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  } } } }

[KevinBaselinesw]

Interesting problem. The python code is doing the equivalent how I wrote the C# code below. In my understanding, each parameter that you pass to the selector (i.e A = stackedobs[..., -3:]) is supposed to correspond to a one of the axis/dimensions. In the python case it looks like you are applying -3: to the second axis. But in reality it seems like .-3: is being applied to the last axis and the ... is being applied to the first 3 axis. I will have to investigate how it is supposed to be.

But you can use the format below to get the desired results.

   def test_ChengYenTang_7(self):

        stackedobs = np.arange(0, 3*12*8*6).reshape(3, 12, 8, 6)
        A = stackedobs[..., -3:]
        print(A)

This produces the results you are looking for!!
        [TestMethod]
        public void test_ChengYenTang_7()
        {
            var stackedobs = np.arange(0, 3*12*8*6).reshape(3, 12, 8, 6);
            var A = stackedobs["...", "...","...","-3:"];
            print(A);
        }

[KevinBaselinesw]

actually this equivalent should be this here:

var A = stackedobs[":", ":", ":", "-3:"];

Strangely enough, I inherited this code from the real numpy team that was trying to build a .NET implementation. The project got cancelled before they could finish but clearly didn't fully understand how ellipsis (...) was supposed to work either.

I will see if I can make this work correctly.

[KevinBaselinesw]

This will also produce the right results.

stackedobs["...", new Slice(-2, null, null)];

FYI, The elipsis "..." is confusing the string parser. I am working on a better solution.

[KevinBaselinesw]

I just put up a new release that should fix both of your recently reported issues.

[ChengYen-Tang]

Thanks, this works fine👍