search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.73k stars 6.81k forks source link

Multiple numpy tests fail with numpy 1.19 #18600

Open eric-haibin-lin opened 4 years ago

eric-haibin-lin commented 4 years ago

The following four numpy op tests fails with numpy==1.19

The failure happens on the master branch CI http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-cpu/detail/master/2034/pipeline and blocks recent PRs http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-cpu/detail/PR-18598/7/pipeline

It only happens recently, and I noticed that the installed numpy version on CI has changed from numpy-1.18.5 to numpy-1.19.0 - see installation log in previous commit and master head. Also notice that the error log shows that the exceptions happen in the _np.xx functions, which is part of the official numpy APIs instead of mx.numpy APIs.

@yzhliu FYI

eric-haibin-lin commented 4 years ago

Error Log 

[2020-06-20T23:26:16.963Z] _______________________________ test_np_delete ________________________________
[2020-06-20T23:26:16.963Z] [gw3] win32 -- Python 3.7.3 C:\Python37\python.exe
[2020-06-20T23:26:16.963Z] 
[2020-06-20T23:26:16.963Z]     @with_seed()
[2020-06-20T23:26:16.963Z]     @use_np
[2020-06-20T23:26:16.963Z]     def test_np_delete():
[2020-06-20T23:26:16.963Z]         class TestDelete(HybridBlock):
[2020-06-20T23:26:16.963Z]             def __init__(self, obj, axis=None):
[2020-06-20T23:26:16.963Z]                 super(TestDelete, self).__init__()
[2020-06-20T23:26:16.963Z]                 self._obj = obj
[2020-06-20T23:26:16.963Z]                 self._axis = axis
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]             def hybrid_forward(self, F, a):
[2020-06-20T23:26:16.963Z]                 return F.np.delete(a, self._obj, axis=self._axis)
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         def GetSize(shp):
[2020-06-20T23:26:16.963Z]             if len(shp) == 0:
[2020-06-20T23:26:16.963Z]                 return 0
[2020-06-20T23:26:16.963Z]             else:
[2020-06-20T23:26:16.963Z]                 res = 1
[2020-06-20T23:26:16.963Z]                 shp_list = list(shp)
[2020-06-20T23:26:16.963Z]                 for x in shp:
[2020-06-20T23:26:16.963Z]                     res *= x
[2020-06-20T23:26:16.963Z]                 return res
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         def GetDimSize(shp, axis):
[2020-06-20T23:26:16.963Z]             if axis is None:
[2020-06-20T23:26:16.963Z]                 return GetSize(shp)
[2020-06-20T23:26:16.963Z]             shp_list = list(shp)
[2020-06-20T23:26:16.963Z]             return shp_list[axis]
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         shape = [(), (0, ), (1, ), (2, 3), (2, 1, 4, 5)]
[2020-06-20T23:26:16.963Z]         config = []
[2020-06-20T23:26:16.963Z]         for shp in shape:
[2020-06-20T23:26:16.963Z]             for ax in range(-1 * len(shp), len(shp), 2):
[2020-06-20T23:26:16.963Z]                 #test slice
[2020-06-20T23:26:16.963Z]                 for st in [-5, -2, 0, 2, 5, None]:
[2020-06-20T23:26:16.963Z]                     for ed in [-5, -2, 0, 2, 5, None]:
[2020-06-20T23:26:16.963Z]                         for stp in [-5, -2, 2, 5, None]:
[2020-06-20T23:26:16.963Z]                             config.append(tuple([shp, slice(st, ed, stp), None]))
[2020-06-20T23:26:16.963Z]                             config.append(tuple([shp, slice(st, ed, stp), ax]))
[2020-06-20T23:26:16.963Z]                 #test iteger
[2020-06-20T23:26:16.963Z]                 for idx in range(-1 * GetDimSize(shp, ax), GetDimSize(shp, ax)):
[2020-06-20T23:26:16.963Z]                     config.append(tuple([shp, idx, ax]))
[2020-06-20T23:26:16.963Z]                 #test ndarray indices
[2020-06-20T23:26:16.963Z]                 idx =  _np.random.randint(-1 * shp[ax], shp[ax] + 1, size = (4)).tolist()
[2020-06-20T23:26:16.963Z]                 config.append(tuple([shp, idx, ax]))
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         for arr_shape, obj, axis in config:
[2020-06-20T23:26:16.963Z]             for objtype in ['int32', 'int64']:
[2020-06-20T23:26:16.963Z]                 if type(obj) == list:
[2020-06-20T23:26:16.963Z]                     obj_mxnp = np.array(obj, dtype=objtype)
[2020-06-20T23:26:16.963Z]                     obj_onp = _np.array(obj, dtype=objtype)
[2020-06-20T23:26:16.963Z]                 elif type(obj) == slice:
[2020-06-20T23:26:16.963Z]                     obj_mxnp = obj
[2020-06-20T23:26:16.963Z]                     obj_onp = obj
[2020-06-20T23:26:16.963Z]                 else:
[2020-06-20T23:26:16.963Z]                     obj_mxnp = (_np.int32(obj) if objtype == 'int32' else _np.int64(obj))
[2020-06-20T23:26:16.963Z]                     obj_onp = (_np.int32(obj) if objtype == 'int32' else _np.int64(obj))
[2020-06-20T23:26:16.963Z]                 test_delete = TestDelete(obj=obj_mxnp, axis=axis)
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]                 a = mx.nd.random.uniform(-1.0, 1.0, shape=arr_shape).as_np_ndarray()
[2020-06-20T23:26:16.963Z]                 a.attach_grad()
[2020-06-20T23:26:16.963Z] >               expected_ret = _np.delete(a.asnumpy(), obj_onp, axis=axis)
[2020-06-20T23:26:16.963Z] 
[2020-06-20T23:26:16.963Z] tests\python\unittest\test_numpy_op.py:4255: 
[2020-06-20T23:26:16.963Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.963Z] <__array_function__ internals>:6: in delete
[2020-06-20T23:26:16.963Z]     ???
[2020-06-20T23:26:16.963Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.963Z] 
[2020-06-20T23:26:16.963Z] arr = array([], dtype=float32), obj = array([0, 0, 0, 0]), axis = 0
[2020-06-20T23:26:16.963Z] 
[2020-06-20T23:26:16.963Z]     @array_function_dispatch(_delete_dispatcher)
[2020-06-20T23:26:16.963Z]     def delete(arr, obj, axis=None):
[2020-06-20T23:26:16.963Z]         """
[2020-06-20T23:26:16.963Z]         Return a new array with sub-arrays along an axis deleted. For a one
[2020-06-20T23:26:16.963Z]         dimensional array, this returns those entries not returned by
[2020-06-20T23:26:16.963Z]         `arr[obj]`.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         Parameters
[2020-06-20T23:26:16.963Z]         ----------
[2020-06-20T23:26:16.963Z]         arr : array_like
[2020-06-20T23:26:16.963Z]             Input array.
[2020-06-20T23:26:16.963Z]         obj : slice, int or array of ints
[2020-06-20T23:26:16.963Z]             Indicate indices of sub-arrays to remove along the specified axis.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]             .. versionchanged:: 1.19.0
[2020-06-20T23:26:16.963Z]                 Boolean indices are now treated as a mask of elements to remove,
[2020-06-20T23:26:16.963Z]                 rather than being cast to the integers 0 and 1.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         axis : int, optional
[2020-06-20T23:26:16.963Z]             The axis along which to delete the subarray defined by `obj`.
[2020-06-20T23:26:16.963Z]             If `axis` is None, `obj` is applied to the flattened array.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         Returns
[2020-06-20T23:26:16.963Z]         -------
[2020-06-20T23:26:16.963Z]         out : ndarray
[2020-06-20T23:26:16.963Z]             A copy of `arr` with the elements specified by `obj` removed. Note
[2020-06-20T23:26:16.963Z]             that `delete` does not occur in-place. If `axis` is None, `out` is
[2020-06-20T23:26:16.963Z]             a flattened array.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         See Also
[2020-06-20T23:26:16.963Z]         --------
[2020-06-20T23:26:16.963Z]         insert : Insert elements into an array.
[2020-06-20T23:26:16.963Z]         append : Append elements at the end of an array.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         Notes
[2020-06-20T23:26:16.963Z]         -----
[2020-06-20T23:26:16.963Z]         Often it is preferable to use a boolean mask. For example:
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         >>> arr = np.arange(12) + 1
[2020-06-20T23:26:16.963Z]         >>> mask = np.ones(len(arr), dtype=bool)
[2020-06-20T23:26:16.963Z]         >>> mask[[0,2,4]] = False
[2020-06-20T23:26:16.963Z]         >>> result = arr[mask,...]
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         Is equivalent to `np.delete(arr, [0,2,4], axis=0)`, but allows further
[2020-06-20T23:26:16.963Z]         use of `mask`.
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         Examples
[2020-06-20T23:26:16.963Z]         --------
[2020-06-20T23:26:16.963Z]         >>> arr = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
[2020-06-20T23:26:16.963Z]         >>> arr
[2020-06-20T23:26:16.963Z]         array([[ 1,  2,  3,  4],
[2020-06-20T23:26:16.963Z]                [ 5,  6,  7,  8],
[2020-06-20T23:26:16.963Z]                [ 9, 10, 11, 12]])
[2020-06-20T23:26:16.963Z]         >>> np.delete(arr, 1, 0)
[2020-06-20T23:26:16.963Z]         array([[ 1,  2,  3,  4],
[2020-06-20T23:26:16.963Z]                [ 9, 10, 11, 12]])
[2020-06-20T23:26:16.963Z]     
[2020-06-20T23:26:16.963Z]         >>> np.delete(arr, np.s_[::2], 1)
[2020-06-20T23:26:16.963Z]         array([[ 2,  4],
[2020-06-20T23:26:16.964Z]                [ 6,  8],
[2020-06-20T23:26:16.964Z]                [10, 12]])
[2020-06-20T23:26:16.964Z]         >>> np.delete(arr, [1,3,5], None)
[2020-06-20T23:26:16.964Z]         array([ 1,  3,  5,  7,  8,  9, 10, 11, 12])
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         """
[2020-06-20T23:26:16.964Z]         wrap = None
[2020-06-20T23:26:16.964Z]         if type(arr) is not ndarray:
[2020-06-20T23:26:16.964Z]             try:
[2020-06-20T23:26:16.964Z]                 wrap = arr.__array_wrap__
[2020-06-20T23:26:16.964Z]             except AttributeError:
[2020-06-20T23:26:16.964Z]                 pass
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         arr = asarray(arr)
[2020-06-20T23:26:16.964Z]         ndim = arr.ndim
[2020-06-20T23:26:16.964Z]         arrorder = 'F' if arr.flags.fnc else 'C'
[2020-06-20T23:26:16.964Z]         if axis is None:
[2020-06-20T23:26:16.964Z]             if ndim != 1:
[2020-06-20T23:26:16.964Z]                 arr = arr.ravel()
[2020-06-20T23:26:16.964Z]             # needed for np.matrix, which is still not 1d after being ravelled
[2020-06-20T23:26:16.964Z]             ndim = arr.ndim
[2020-06-20T23:26:16.964Z]             axis = ndim - 1
[2020-06-20T23:26:16.964Z]         else:
[2020-06-20T23:26:16.964Z]             axis = normalize_axis_index(axis, ndim)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         slobj = [slice(None)]*ndim
[2020-06-20T23:26:16.964Z]         N = arr.shape[axis]
[2020-06-20T23:26:16.964Z]         newshape = list(arr.shape)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         if isinstance(obj, slice):
[2020-06-20T23:26:16.964Z]             start, stop, step = obj.indices(N)
[2020-06-20T23:26:16.964Z]             xr = range(start, stop, step)
[2020-06-20T23:26:16.964Z]             numtodel = len(xr)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             if numtodel <= 0:
[2020-06-20T23:26:16.964Z]                 if wrap:
[2020-06-20T23:26:16.964Z]                     return wrap(arr.copy(order=arrorder))
[2020-06-20T23:26:16.964Z]                 else:
[2020-06-20T23:26:16.964Z]                     return arr.copy(order=arrorder)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             # Invert if step is negative:
[2020-06-20T23:26:16.964Z]             if step < 0:
[2020-06-20T23:26:16.964Z]                 step = -step
[2020-06-20T23:26:16.964Z]                 start = xr[-1]
[2020-06-20T23:26:16.964Z]                 stop = xr[0] + 1
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             newshape[axis] -= numtodel
[2020-06-20T23:26:16.964Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-20T23:26:16.964Z]             # copy initial chunk
[2020-06-20T23:26:16.964Z]             if start == 0:
[2020-06-20T23:26:16.964Z]                 pass
[2020-06-20T23:26:16.964Z]             else:
[2020-06-20T23:26:16.964Z]                 slobj[axis] = slice(None, start)
[2020-06-20T23:26:16.964Z]                 new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-20T23:26:16.964Z]             # copy end chunk
[2020-06-20T23:26:16.964Z]             if stop == N:
[2020-06-20T23:26:16.964Z]                 pass
[2020-06-20T23:26:16.964Z]             else:
[2020-06-20T23:26:16.964Z]                 slobj[axis] = slice(stop-numtodel, None)
[2020-06-20T23:26:16.964Z]                 slobj2 = [slice(None)]*ndim
[2020-06-20T23:26:16.964Z]                 slobj2[axis] = slice(stop, None)
[2020-06-20T23:26:16.964Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-20T23:26:16.964Z]             # copy middle pieces
[2020-06-20T23:26:16.964Z]             if step == 1:
[2020-06-20T23:26:16.964Z]                 pass
[2020-06-20T23:26:16.964Z]             else:  # use array indexing.
[2020-06-20T23:26:16.964Z]                 keep = ones(stop-start, dtype=bool)
[2020-06-20T23:26:16.964Z]                 keep[:stop-start:step] = False
[2020-06-20T23:26:16.964Z]                 slobj[axis] = slice(start, stop-numtodel)
[2020-06-20T23:26:16.964Z]                 slobj2 = [slice(None)]*ndim
[2020-06-20T23:26:16.964Z]                 slobj2[axis] = slice(start, stop)
[2020-06-20T23:26:16.964Z]                 arr = arr[tuple(slobj2)]
[2020-06-20T23:26:16.964Z]                 slobj2[axis] = keep
[2020-06-20T23:26:16.964Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-20T23:26:16.964Z]             if wrap:
[2020-06-20T23:26:16.964Z]                 return wrap(new)
[2020-06-20T23:26:16.964Z]             else:
[2020-06-20T23:26:16.964Z]                 return new
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         if isinstance(obj, (int, integer)) and not isinstance(obj, bool):
[2020-06-20T23:26:16.964Z]             # optimization for a single value
[2020-06-20T23:26:16.964Z]             if (obj < -N or obj >= N):
[2020-06-20T23:26:16.964Z]                 raise IndexError(
[2020-06-20T23:26:16.964Z]                     "index %i is out of bounds for axis %i with "
[2020-06-20T23:26:16.964Z]                     "size %i" % (obj, axis, N))
[2020-06-20T23:26:16.964Z]             if (obj < 0):
[2020-06-20T23:26:16.964Z]                 obj += N
[2020-06-20T23:26:16.964Z]             newshape[axis] -= 1
[2020-06-20T23:26:16.964Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-20T23:26:16.964Z]             slobj[axis] = slice(None, obj)
[2020-06-20T23:26:16.964Z]             new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-20T23:26:16.964Z]             slobj[axis] = slice(obj, None)
[2020-06-20T23:26:16.964Z]             slobj2 = [slice(None)]*ndim
[2020-06-20T23:26:16.964Z]             slobj2[axis] = slice(obj+1, None)
[2020-06-20T23:26:16.964Z]             new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-20T23:26:16.964Z]         else:
[2020-06-20T23:26:16.964Z]             _obj = obj
[2020-06-20T23:26:16.964Z]             obj = np.asarray(obj)
[2020-06-20T23:26:16.964Z]             if obj.size == 0 and not isinstance(_obj, np.ndarray):
[2020-06-20T23:26:16.964Z]                 obj = obj.astype(intp)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             if obj.dtype == bool:
[2020-06-20T23:26:16.964Z]                 if obj.shape != (N,):
[2020-06-20T23:26:16.964Z]                     raise ValueError('boolean array argument obj to delete '
[2020-06-20T23:26:16.964Z]                                      'must be one dimensional and match the axis '
[2020-06-20T23:26:16.964Z]                                      'length of {}'.format(N))
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]                 # optimization, the other branch is slower
[2020-06-20T23:26:16.964Z]                 keep = ~obj
[2020-06-20T23:26:16.964Z]             else:
[2020-06-20T23:26:16.964Z]                 keep = ones(N, dtype=bool)
[2020-06-20T23:26:16.964Z] >               keep[obj,] = False
[2020-06-20T23:26:16.964Z] E               IndexError: index 0 is out of bounds for axis 0 with size 0
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] C:\Python37\lib\site-packages\numpy\lib\function_base.py:4406: IndexError
[2020-06-20T23:26:16.964Z] ----------------------------- Captured log setup ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    root:conftest.py:193 np/mx/python random seeds are set to 1481650552, use MXNET_TEST_SEED=1481650552 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured stderr call -----------------------------
[2020-06-20T23:26:16.964Z] [DEBUG] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=117564977 to reproduce.
[2020-06-20T23:26:16.964Z] [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=117564977 to reproduce.
[2020-06-20T23:26:16.964Z] ------------------------------ Captured log call ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    common:common.py:221 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=117564977 to reproduce.
[2020-06-20T23:26:16.964Z] INFO     common:common.py:227 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=117564977 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured log teardown ----------------------------
[2020-06-20T23:26:16.964Z] INFO     root:conftest.py:210 np/mx/python random seeds are set to 1481650552, use MXNET_TEST_SEED=1481650552 to reproduce.
[2020-06-20T23:26:16.964Z] _____________________________ test_np_random_beta _____________________________
[2020-06-20T23:26:16.964Z] [gw3] win32 -- Python 3.7.3 C:\Python37\python.exe
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z]     @with_seed()
[2020-06-20T23:26:16.964Z]     @use_np
[2020-06-20T23:26:16.964Z]     def test_np_random_beta():
[2020-06-20T23:26:16.964Z]         class TestRandomBeta(HybridBlock):
[2020-06-20T23:26:16.964Z]             def __init__(self, size=None, dtype=None, ctx=None):
[2020-06-20T23:26:16.964Z]                 super(TestRandomBeta, self).__init__()
[2020-06-20T23:26:16.964Z]                 self._size = size
[2020-06-20T23:26:16.964Z]                 self._dtype = dtype
[2020-06-20T23:26:16.964Z]                 self._ctx = ctx
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             def hybrid_forward(self, F, a, b):
[2020-06-20T23:26:16.964Z]                 return F.np.random.beta(a, b, size=self._size, dtype=self._dtype, ctx=self._ctx)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         def _test_random_beta_range(output):
[2020-06-20T23:26:16.964Z]             bigger_than_zero = _np.all(output > 0)
[2020-06-20T23:26:16.964Z]             smaller_than_one = _np.all(output < 1)
[2020-06-20T23:26:16.964Z]             return bigger_than_zero and smaller_than_one
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         shape_list = [(), (1,), (2, 3), (4, 0, 5), 6, (7, 8), None]
[2020-06-20T23:26:16.964Z]         # since fp16 might incur precision issue, the corresponding test is skipped
[2020-06-20T23:26:16.964Z]         dtype_list = [np.float32, np.float64]
[2020-06-20T23:26:16.964Z]         hybridize_list = [False, True]
[2020-06-20T23:26:16.964Z]         data = np.array([1])
[2020-06-20T23:26:16.964Z]         for [param_shape, in_dtype, out_dtype, hybridize] in itertools.product(shape_list,
[2020-06-20T23:26:16.964Z]                 dtype_list, dtype_list, hybridize_list):
[2020-06-20T23:26:16.964Z]             mx_data = data.astype(in_dtype)
[2020-06-20T23:26:16.964Z]             np_data = mx_data.asnumpy()
[2020-06-20T23:26:16.964Z]             test_random_beta = TestRandomBeta(size=param_shape, dtype=out_dtype)
[2020-06-20T23:26:16.964Z]             if hybridize:
[2020-06-20T23:26:16.964Z]                 test_random_beta.hybridize()
[2020-06-20T23:26:16.964Z] >           np_out = _np.random.beta(np_data, np_data, size=param_shape)
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] tests\python\unittest\test_numpy_op.py:4786: 
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] mtrand.pyx:478: in numpy.random.mtrand.RandomState.beta
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:541: in numpy.random._common.cont
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:459: in numpy.random._common.cont_broadcast_2
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] >   ???
[2020-06-20T23:26:16.964Z] E   ValueError: Output size () is not compatible with broadcast dimensions of inputs (1,).
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] _common.pyx:229: ValueError
[2020-06-20T23:26:16.964Z] ----------------------------- Captured log setup ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    root:conftest.py:193 np/mx/python random seeds are set to 404557395, use MXNET_TEST_SEED=404557395 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured stderr call -----------------------------
[2020-06-20T23:26:16.964Z] [DEBUG] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1213699728 to reproduce.
[2020-06-20T23:26:16.964Z] [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1213699728 to reproduce.
[2020-06-20T23:26:16.964Z] ------------------------------ Captured log call ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    common:common.py:221 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1213699728 to reproduce.
[2020-06-20T23:26:16.964Z] INFO     common:common.py:227 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1213699728 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured log teardown ----------------------------
[2020-06-20T23:26:16.964Z] INFO     root:conftest.py:210 np/mx/python random seeds are set to 404557395, use MXNET_TEST_SEED=404557395 to reproduce.
[2020-06-20T23:26:16.964Z] ______________________________ test_np_random_f _______________________________
[2020-06-20T23:26:16.964Z] [gw3] win32 -- Python 3.7.3 C:\Python37\python.exe
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z]     @with_seed()
[2020-06-20T23:26:16.964Z]     @use_np
[2020-06-20T23:26:16.964Z]     def test_np_random_f():
[2020-06-20T23:26:16.964Z]         class TestRandomF(HybridBlock):
[2020-06-20T23:26:16.964Z]             def __init__(self, size=None):
[2020-06-20T23:26:16.964Z]                 super(TestRandomF, self).__init__()
[2020-06-20T23:26:16.964Z]                 self._size = size
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             def hybrid_forward(self, F, dfnum, dfden):
[2020-06-20T23:26:16.964Z]                 return F.np.random.f(dfnum, dfden, size=self._size)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         shape_list = [(), (1,), (2, 3), (4, 0, 5), 6, (7, 8), None]
[2020-06-20T23:26:16.964Z]         hybridize_list = [False, True]
[2020-06-20T23:26:16.964Z]         df = np.array([1])
[2020-06-20T23:26:16.964Z]         for [param_shape, hybridize] in itertools.product(shape_list,
[2020-06-20T23:26:16.964Z]              hybridize_list):
[2020-06-20T23:26:16.964Z]             if sys.version_info.major < 3 and param_shape == ():
[2020-06-20T23:26:16.964Z]                 continue
[2020-06-20T23:26:16.964Z]             mx_df = df
[2020-06-20T23:26:16.964Z]             np_df = mx_df.asnumpy()
[2020-06-20T23:26:16.964Z]             test_random_f = TestRandomF(size=param_shape)
[2020-06-20T23:26:16.964Z]             if hybridize:
[2020-06-20T23:26:16.964Z]                 test_random_f.hybridize()
[2020-06-20T23:26:16.964Z] >           np_out = _np.random.f(np_df, np_df, size=param_shape)
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] tests\python\unittest\test_numpy_op.py:4823: 
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] mtrand.pyx:1753: in numpy.random.mtrand.RandomState.f
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:541: in numpy.random._common.cont
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:459: in numpy.random._common.cont_broadcast_2
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] >   ???
[2020-06-20T23:26:16.964Z] E   ValueError: Output size () is not compatible with broadcast dimensions of inputs (1,).
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] _common.pyx:229: ValueError
[2020-06-20T23:26:16.964Z] ----------------------------- Captured log setup ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    root:conftest.py:193 np/mx/python random seeds are set to 364715442, use MXNET_TEST_SEED=364715442 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured stderr call -----------------------------
[2020-06-20T23:26:16.964Z] [DEBUG] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=867177126 to reproduce.
[2020-06-20T23:26:16.964Z] [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=867177126 to reproduce.
[2020-06-20T23:26:16.964Z] ------------------------------ Captured log call ------------------------------
[2020-06-20T23:26:16.964Z] DEBUG    common:common.py:221 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=867177126 to reproduce.
[2020-06-20T23:26:16.964Z] INFO     common:common.py:227 Setting test np/mx/python random seeds, use MXNET_TEST_SEED=867177126 to reproduce.
[2020-06-20T23:26:16.964Z] ---------------------------- Captured log teardown ----------------------------
[2020-06-20T23:26:16.964Z] INFO     root:conftest.py:210 np/mx/python random seeds are set to 364715442, use MXNET_TEST_SEED=364715442 to reproduce.
[2020-06-20T23:26:16.964Z] __________________________ test_np_random_chisquare ___________________________
[2020-06-20T23:26:16.964Z] [gw3] win32 -- Python 3.7.3 C:\Python37\python.exe
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z]     @with_seed()
[2020-06-20T23:26:16.964Z]     @use_np
[2020-06-20T23:26:16.964Z]     def test_np_random_chisquare():
[2020-06-20T23:26:16.964Z]         class TestRandomChisquare(HybridBlock):
[2020-06-20T23:26:16.964Z]             def __init__(self, size=None, dtype=None, ctx=None):
[2020-06-20T23:26:16.964Z]                 super(TestRandomChisquare, self).__init__()
[2020-06-20T23:26:16.964Z]                 self._size = size
[2020-06-20T23:26:16.964Z]                 self._dtype = dtype
[2020-06-20T23:26:16.964Z]                 self._ctx = ctx
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]             def hybrid_forward(self, F, df):
[2020-06-20T23:26:16.964Z]                 return F.np.random.chisquare(df, size=self._size, dtype=self._dtype, ctx=self._ctx)
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         shape_list = [(), (1,), (2, 3), (4, 0, 5), 6, (7, 8), None]
[2020-06-20T23:26:16.964Z]     
[2020-06-20T23:26:16.964Z]         dtype_list = [np.float16, np.float32, np.float64]
[2020-06-20T23:26:16.964Z]         hybridize_list = [False, True]
[2020-06-20T23:26:16.964Z]         df = np.array([1])
[2020-06-20T23:26:16.964Z]         for [param_shape, in_dtype, out_dtype, hybridize] in itertools.product(shape_list,
[2020-06-20T23:26:16.964Z]                 dtype_list, dtype_list, hybridize_list):
[2020-06-20T23:26:16.964Z]             if sys.version_info.major < 3 and param_shape == ():
[2020-06-20T23:26:16.964Z]                 continue
[2020-06-20T23:26:16.964Z]             mx_df = df.astype(in_dtype)
[2020-06-20T23:26:16.964Z]             np_df = mx_df.asnumpy()
[2020-06-20T23:26:16.964Z]             test_random_chisquare = TestRandomChisquare(size=param_shape, dtype=out_dtype)
[2020-06-20T23:26:16.964Z]             if hybridize:
[2020-06-20T23:26:16.964Z]                 test_random_chisquare.hybridize()
[2020-06-20T23:26:16.964Z] >           np_out = _np.random.chisquare(np_df, size=param_shape)
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] tests\python\unittest\test_numpy_op.py:4858: 
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] mtrand.pyx:1913: in numpy.random.mtrand.RandomState.chisquare
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:537: in numpy.random._common.cont
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _common.pyx:421: in numpy.random._common.cont_broadcast_1
[2020-06-20T23:26:16.964Z]     ???
[2020-06-20T23:26:16.964Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] >   ???
[2020-06-20T23:26:16.964Z] E   ValueError: Output size () is not compatible with broadcast dimensions of inputs (1,).
[2020-06-20T23:26:16.964Z] 
[2020-06-20T23:26:16.964Z] _common.pyx:229: ValueError
[2020-06-20T23:26:16.964Z]
eric-haibin-lin commented 4 years ago

np.delete also fails the compatibility test:

[2020-06-22T00:36:45.758Z] ================================== FAILURES ===================================
[2020-06-22T00:36:45.758Z] _______________________ test_np_array_function_protocol _______________________
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] args = (), kwargs = {}
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     @functools.wraps(func)
[2020-06-22T00:36:45.758Z]     def _run_with_array_func_proto(*args, **kwargs):
[2020-06-22T00:36:45.758Z]         if cur_np_ver >= np_1_17_ver:
[2020-06-22T00:36:45.758Z]             try:
[2020-06-22T00:36:45.758Z] >               func(*args, **kwargs)
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] windows_package\python\mxnet\numpy_dispatch_protocol.py:55: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     @with_seed()
[2020-06-22T00:36:45.758Z]     @use_np
[2020-06-22T00:36:45.758Z]     @with_array_function_protocol
[2020-06-22T00:36:45.758Z]     @pytest.mark.serial
[2020-06-22T00:36:45.758Z]     def test_np_array_function_protocol():
[2020-06-22T00:36:45.758Z] >       check_interoperability(_NUMPY_ARRAY_FUNCTION_LIST)
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] tests\python\unittest\test_numpy_interoperability.py:3262: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] op_list = ['all', 'any', 'sometrue', 'argmin', 'argmax', 'around', ...]
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     def check_interoperability(op_list):
[2020-06-22T00:36:45.758Z]         for name in op_list:
[2020-06-22T00:36:45.758Z]             if name in _TVM_OPS and not is_op_runnable():
[2020-06-22T00:36:45.758Z]                 continue
[2020-06-22T00:36:45.758Z]             if name in ['shares_memory', 'may_share_memory', 'empty_like',
[2020-06-22T00:36:45.758Z]                         '__version__', 'dtype', '_NoValue']:  # skip list
[2020-06-22T00:36:45.758Z]                 continue
[2020-06-22T00:36:45.758Z]             if name in ['full_like', 'zeros_like', 'ones_like'] and \
[2020-06-22T00:36:45.758Z]                     StrictVersion(platform.python_version()) < StrictVersion('3.0.0'):
[2020-06-22T00:36:45.758Z]                 continue
[2020-06-22T00:36:45.758Z]             print('Dispatch test:', name)
[2020-06-22T00:36:45.758Z]             workloads = OpArgMngr.get_workloads(name)
[2020-06-22T00:36:45.758Z]             assert workloads is not None, 'Workloads for operator `{}` has not been ' \
[2020-06-22T00:36:45.758Z]                                           'added for checking interoperability with ' \
[2020-06-22T00:36:45.758Z]                                           'the official NumPy.'.format(name)
[2020-06-22T00:36:45.758Z]             for workload in workloads:
[2020-06-22T00:36:45.758Z] >               _check_interoperability_helper(name, *workload['args'], **workload['kwargs'])
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] tests\python\unittest\test_numpy_interoperability.py:3240: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] op_name = 'delete', args = (array([0., 1., 2., 3., 4.]), array([]))
[2020-06-22T00:36:45.758Z] kwargs = {'axis': 0}, strs = ['delete']
[2020-06-22T00:36:45.758Z] onp_op = <function delete at 0x000002C47E8C7158>
[2020-06-22T00:36:45.758Z] out = array([0., 1., 2., 3., 4.])
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     def _check_interoperability_helper(op_name, *args, **kwargs):
[2020-06-22T00:36:45.758Z]         strs = op_name.split('.')
[2020-06-22T00:36:45.758Z]         if len(strs) == 1:
[2020-06-22T00:36:45.758Z]             onp_op = getattr(_np, op_name)
[2020-06-22T00:36:45.758Z]         elif len(strs) == 2:
[2020-06-22T00:36:45.758Z]             onp_op = getattr(getattr(_np, strs[0]), strs[1])
[2020-06-22T00:36:45.758Z]         else:
[2020-06-22T00:36:45.758Z]             assert False
[2020-06-22T00:36:45.758Z]         if not is_op_runnable():
[2020-06-22T00:36:45.758Z]             return
[2020-06-22T00:36:45.758Z]         out = onp_op(*args, **kwargs)
[2020-06-22T00:36:45.758Z] >       expected_out = _get_numpy_op_output(onp_op, *args, **kwargs)
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] tests\python\unittest\test_numpy_interoperability.py:3204: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] onp_op = <function delete at 0x000002C47E8C7158>
[2020-06-22T00:36:45.758Z] args = (array([0., 1., 2., 3., 4.]), array([])), kwargs = {'axis': 0}
[2020-06-22T00:36:45.758Z] onp_args = [array([0., 1., 2., 3., 4.], dtype=float32), array([], dtype=float32)]
[2020-06-22T00:36:45.758Z] onp_kwargs = {'axis': 0}, i = 1, v = array([], dtype=float32)
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     def _get_numpy_op_output(onp_op, *args, **kwargs):
[2020-06-22T00:36:45.758Z]         onp_args = [arg.asnumpy() if isinstance(arg, np.ndarray) else arg for arg in args]
[2020-06-22T00:36:45.758Z]         onp_kwargs = {k: v.asnumpy() if isinstance(v, np.ndarray) else v for k, v in kwargs.items()}
[2020-06-22T00:36:45.758Z]         for i, v in enumerate(onp_args):
[2020-06-22T00:36:45.758Z]             if isinstance(v, (list, tuple)):
[2020-06-22T00:36:45.758Z]                 new_arrs = [a.asnumpy() if isinstance(a, np.ndarray) else a for a in v]
[2020-06-22T00:36:45.758Z]                 onp_args[i] = new_arrs
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z] >       return onp_op(*onp_args, **onp_kwargs)
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] tests\python\unittest\test_numpy_interoperability.py:3190: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] args = (array([0., 1., 2., 3., 4.], dtype=float32), array([], dtype=float32))
[2020-06-22T00:36:45.758Z] kwargs = {'axis': 0}
[2020-06-22T00:36:45.758Z] relevant_args = (array([0., 1., 2., 3., 4.], dtype=float32), array([], dtype=float32))
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] >   ???
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] <__array_function__ internals>:6: 
[2020-06-22T00:36:45.758Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z] arr = array([0., 1., 2., 3., 4.], dtype=float32), obj = array([], dtype=float32)
[2020-06-22T00:36:45.758Z] axis = 0
[2020-06-22T00:36:45.758Z] 
[2020-06-22T00:36:45.758Z]     @array_function_dispatch(_delete_dispatcher)
[2020-06-22T00:36:45.758Z]     def delete(arr, obj, axis=None):
[2020-06-22T00:36:45.758Z]         """
[2020-06-22T00:36:45.758Z]         Return a new array with sub-arrays along an axis deleted. For a one
[2020-06-22T00:36:45.758Z]         dimensional array, this returns those entries not returned by
[2020-06-22T00:36:45.758Z]         `arr[obj]`.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         Parameters
[2020-06-22T00:36:45.758Z]         ----------
[2020-06-22T00:36:45.758Z]         arr : array_like
[2020-06-22T00:36:45.758Z]             Input array.
[2020-06-22T00:36:45.758Z]         obj : slice, int or array of ints
[2020-06-22T00:36:45.758Z]             Indicate indices of sub-arrays to remove along the specified axis.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]             .. versionchanged:: 1.19.0
[2020-06-22T00:36:45.758Z]                 Boolean indices are now treated as a mask of elements to remove,
[2020-06-22T00:36:45.758Z]                 rather than being cast to the integers 0 and 1.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         axis : int, optional
[2020-06-22T00:36:45.758Z]             The axis along which to delete the subarray defined by `obj`.
[2020-06-22T00:36:45.758Z]             If `axis` is None, `obj` is applied to the flattened array.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         Returns
[2020-06-22T00:36:45.758Z]         -------
[2020-06-22T00:36:45.758Z]         out : ndarray
[2020-06-22T00:36:45.758Z]             A copy of `arr` with the elements specified by `obj` removed. Note
[2020-06-22T00:36:45.758Z]             that `delete` does not occur in-place. If `axis` is None, `out` is
[2020-06-22T00:36:45.758Z]             a flattened array.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         See Also
[2020-06-22T00:36:45.758Z]         --------
[2020-06-22T00:36:45.758Z]         insert : Insert elements into an array.
[2020-06-22T00:36:45.758Z]         append : Append elements at the end of an array.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         Notes
[2020-06-22T00:36:45.758Z]         -----
[2020-06-22T00:36:45.758Z]         Often it is preferable to use a boolean mask. For example:
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         >>> arr = np.arange(12) + 1
[2020-06-22T00:36:45.758Z]         >>> mask = np.ones(len(arr), dtype=bool)
[2020-06-22T00:36:45.758Z]         >>> mask[[0,2,4]] = False
[2020-06-22T00:36:45.758Z]         >>> result = arr[mask,...]
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         Is equivalent to `np.delete(arr, [0,2,4], axis=0)`, but allows further
[2020-06-22T00:36:45.758Z]         use of `mask`.
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         Examples
[2020-06-22T00:36:45.758Z]         --------
[2020-06-22T00:36:45.758Z]         >>> arr = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
[2020-06-22T00:36:45.758Z]         >>> arr
[2020-06-22T00:36:45.758Z]         array([[ 1,  2,  3,  4],
[2020-06-22T00:36:45.758Z]                [ 5,  6,  7,  8],
[2020-06-22T00:36:45.758Z]                [ 9, 10, 11, 12]])
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, 1, 0)
[2020-06-22T00:36:45.758Z]         array([[ 1,  2,  3,  4],
[2020-06-22T00:36:45.758Z]                [ 9, 10, 11, 12]])
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, np.s_[::2], 1)
[2020-06-22T00:36:45.758Z]         array([[ 2,  4],
[2020-06-22T00:36:45.758Z]                [ 6,  8],
[2020-06-22T00:36:45.758Z]                [10, 12]])
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, [1,3,5], None)
[2020-06-22T00:36:45.758Z]         array([ 1,  3,  5,  7,  8,  9, 10, 11, 12])
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         """
[2020-06-22T00:36:45.758Z]         wrap = None
[2020-06-22T00:36:45.758Z]         if type(arr) is not ndarray:
[2020-06-22T00:36:45.758Z]             try:
[2020-06-22T00:36:45.758Z]                 wrap = arr.__array_wrap__
[2020-06-22T00:36:45.758Z]             except AttributeError:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         arr = asarray(arr)
[2020-06-22T00:36:45.758Z]         ndim = arr.ndim
[2020-06-22T00:36:45.758Z]         arrorder = 'F' if arr.flags.fnc else 'C'
[2020-06-22T00:36:45.758Z]         if axis is None:
[2020-06-22T00:36:45.758Z]             if ndim != 1:
[2020-06-22T00:36:45.758Z]                 arr = arr.ravel()
[2020-06-22T00:36:45.758Z]             # needed for np.matrix, which is still not 1d after being ravelled
[2020-06-22T00:36:45.758Z]             ndim = arr.ndim
[2020-06-22T00:36:45.758Z]             axis = ndim - 1
[2020-06-22T00:36:45.758Z]         else:
[2020-06-22T00:36:45.758Z]             axis = normalize_axis_index(axis, ndim)
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         slobj = [slice(None)]*ndim
[2020-06-22T00:36:45.758Z]         N = arr.shape[axis]
[2020-06-22T00:36:45.758Z]         newshape = list(arr.shape)
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]         if isinstance(obj, slice):
[2020-06-22T00:36:45.758Z]             start, stop, step = obj.indices(N)
[2020-06-22T00:36:45.758Z]             xr = range(start, stop, step)
[2020-06-22T00:36:45.758Z]             numtodel = len(xr)
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]             if numtodel <= 0:
[2020-06-22T00:36:45.758Z]                 if wrap:
[2020-06-22T00:36:45.758Z]                     return wrap(arr.copy(order=arrorder))
[2020-06-22T00:36:45.758Z]                 else:
[2020-06-22T00:36:45.758Z]                     return arr.copy(order=arrorder)
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]             # Invert if step is negative:
[2020-06-22T00:36:45.758Z]             if step < 0:
[2020-06-22T00:36:45.758Z]                 step = -step
[2020-06-22T00:36:45.758Z]                 start = xr[-1]
[2020-06-22T00:36:45.758Z]                 stop = xr[0] + 1
[2020-06-22T00:36:45.758Z]     
[2020-06-22T00:36:45.758Z]             newshape[axis] -= numtodel
[2020-06-22T00:36:45.758Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-22T00:36:45.758Z]             # copy initial chunk
[2020-06-22T00:36:45.758Z]             if start == 0:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.758Z]             else:
[2020-06-22T00:36:45.758Z]                 slobj[axis] = slice(None, start)
[2020-06-22T00:36:45.758Z]                 new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-22T00:36:45.758Z]             # copy end chunk
[2020-06-22T00:36:45.758Z]             if stop == N:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 slobj[axis] = slice(stop-numtodel, None)
[2020-06-22T00:36:45.759Z]                 slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = slice(stop, None)
[2020-06-22T00:36:45.759Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]             # copy middle pieces
[2020-06-22T00:36:45.759Z]             if step == 1:
[2020-06-22T00:36:45.759Z]                 pass
[2020-06-22T00:36:45.759Z]             else:  # use array indexing.
[2020-06-22T00:36:45.759Z]                 keep = ones(stop-start, dtype=bool)
[2020-06-22T00:36:45.759Z]                 keep[:stop-start:step] = False
[2020-06-22T00:36:45.759Z]                 slobj[axis] = slice(start, stop-numtodel)
[2020-06-22T00:36:45.759Z]                 slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = slice(start, stop)
[2020-06-22T00:36:45.759Z]                 arr = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = keep
[2020-06-22T00:36:45.759Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]             if wrap:
[2020-06-22T00:36:45.759Z]                 return wrap(new)
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 return new
[2020-06-22T00:36:45.759Z]     
[2020-06-22T00:36:45.759Z]         if isinstance(obj, (int, integer)) and not isinstance(obj, bool):
[2020-06-22T00:36:45.759Z]             # optimization for a single value
[2020-06-22T00:36:45.759Z]             if (obj < -N or obj >= N):
[2020-06-22T00:36:45.759Z]                 raise IndexError(
[2020-06-22T00:36:45.759Z]                     "index %i is out of bounds for axis %i with "
[2020-06-22T00:36:45.759Z]                     "size %i" % (obj, axis, N))
[2020-06-22T00:36:45.759Z]             if (obj < 0):
[2020-06-22T00:36:45.759Z]                 obj += N
[2020-06-22T00:36:45.759Z]             newshape[axis] -= 1
[2020-06-22T00:36:45.759Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-22T00:36:45.759Z]             slobj[axis] = slice(None, obj)
[2020-06-22T00:36:45.759Z]             new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-22T00:36:45.759Z]             slobj[axis] = slice(obj, None)
[2020-06-22T00:36:45.759Z]             slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]             slobj2[axis] = slice(obj+1, None)
[2020-06-22T00:36:45.759Z]             new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]         else:
[2020-06-22T00:36:45.759Z]             _obj = obj
[2020-06-22T00:36:45.759Z]             obj = np.asarray(obj)
[2020-06-22T00:36:45.759Z]             if obj.size == 0 and not isinstance(_obj, np.ndarray):
[2020-06-22T00:36:45.759Z]                 obj = obj.astype(intp)
[2020-06-22T00:36:45.759Z]     
[2020-06-22T00:36:45.759Z]             if obj.dtype == bool:
[2020-06-22T00:36:45.759Z]                 if obj.shape != (N,):
[2020-06-22T00:36:45.759Z]                     raise ValueError('boolean array argument obj to delete '
[2020-06-22T00:36:45.759Z]                                      'must be one dimensional and match the axis '
[2020-06-22T00:36:45.759Z]                                      'length of {}'.format(N))
[2020-06-22T00:36:45.759Z]     
[2020-06-22T00:36:45.759Z]                 # optimization, the other branch is slower
[2020-06-22T00:36:45.759Z]                 keep = ~obj
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 keep = ones(N, dtype=bool)
[2020-06-22T00:36:45.759Z] >               keep[obj,] = False
[2020-06-22T00:36:45.759Z] E               IndexError: arrays used as indices must be of integer (or boolean) type
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z] C:\Python37\lib\site-packages\numpy\lib\function_base.py:4406: IndexError
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z] During handling of the above exception, another exception occurred:
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z] args = (), kwargs = {}, test_count = 1, env_seed_str = None, i = 0
[2020-06-22T00:36:45.759Z] this_test_seed = 810322850, log_level = 10
[2020-06-22T00:36:45.759Z] post_test_state = ('MT19937', array([1938474601, 2158246084, 2260818444,   19860296, 1648467405,
[2020-06-22T00:36:45.759Z]        3604725506,  725127881, 21902135..., 1802122558,  653485402, 3067916522,
[2020-06-22T00:36:45.759Z]        1738520349,  675050936, 3660368646, 2725207562], dtype=uint32), 1, 0, 0.0)
[2020-06-22T00:36:45.759Z] logger = <Logger common (DEBUG)>, test_count_msg = ''
[2020-06-22T00:36:45.759Z] test_msg = 'Setting test np/mx/python random seeds, use MXNET_TEST_SEED=810322850 to reproduce.'
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z]     @functools.wraps(orig_test)
[2020-06-22T00:36:45.759Z]     def test_new(*args, **kwargs):
[2020-06-22T00:36:45.759Z]         test_count = int(os.getenv('MXNET_TEST_COUNT', '1'))
[2020-06-22T00:36:45.759Z]         env_seed_str = os.getenv('MXNET_TEST_SEED')
[2020-06-22T00:36:45.759Z]         for i in range(test_count):
[2020-06-22T00:36:45.759Z]             if seed is not None:
[2020-06-22T00:36:45.759Z]                 this_test_seed = seed
[2020-06-22T00:36:45.759Z]                 log_level = logging.INFO
[2020-06-22T00:36:45.759Z]             elif env_seed_str is not None:
[2020-06-22T00:36:45.759Z]                 this_test_seed = int(env_seed_str)
[2020-06-22T00:36:45.759Z]                 log_level = logging.INFO
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 this_test_seed = np.random.randint(0, np.iinfo(np.int32).max)
[2020-06-22T00:36:45.759Z]                 log_level = logging.DEBUG
[2020-06-22T00:36:45.759Z]             post_test_state = np.random.get_state()
[2020-06-22T00:36:45.759Z]             np.random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             mx.random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             logger = default_logger()
[2020-06-22T00:36:45.759Z]             # 'pytest --logging-level=DEBUG' shows this msg even with an ensuing core dump.
[2020-06-22T00:36:45.759Z]             test_count_msg = '{} of {}: '.format(i+1,test_count) if test_count > 1 else ''
[2020-06-22T00:36:45.759Z]             test_msg = ('{}Setting test np/mx/python random seeds, use MXNET_TEST_SEED={}'
[2020-06-22T00:36:45.759Z]                         ' to reproduce.').format(test_count_msg, this_test_seed)
[2020-06-22T00:36:45.759Z]             logger.log(log_level, test_msg)
[2020-06-22T00:36:45.759Z]             try:
[2020-06-22T00:36:45.759Z] >               orig_test(*args, **kwargs)
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z] tests\python\unittest\common.py:223: 
[2020-06-22T00:36:45.759Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.759Z] windows_package\python\mxnet\util.py:298: in _with_np_shape
[2020-06-22T00:36:45.759Z]     return func(*args, **kwargs)
[2020-06-22T00:36:45.759Z] windows_package\python\mxnet\util.py:482: in _with_np_array
[2020-06-22T00:36:45.759Z]     return func(*args, **kwargs)
[2020-06-22T00:36:45.759Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z] args = (), kwargs = {}
[2020-06-22T00:36:45.759Z] 
[2020-06-22T00:36:45.759Z]     @functools.wraps(func)
[2020-06-22T00:36:45.759Z]     def _run_with_array_func_proto(*args, **kwargs):
[2020-06-22T00:36:45.759Z]         if cur_np_ver >= np_1_17_ver:
[2020-06-22T00:36:45.759Z]             try:
[2020-06-22T00:36:45.759Z]                 func(*args, **kwargs)
[2020-06-22T00:36:45.759Z]             except Exception as e:
[2020-06-22T00:36:45.759Z]                 raise RuntimeError('Running function {} with NumPy array function protocol failed'
[2020-06-22T00:36:45.759Z]                                    ' with exception {}'
[2020-06-22T00:36:45.759Z] >                                  .format(func.__name__, str(e)))
[2020-06-22T00:36:45.759Z] E               RuntimeError: Running function test_np_array_function_protocol with NumPy array function protocol failed with exception arrays used as indices must be of integer (or boolean) type
eric-haibin-lin commented 4 years ago

Temporarily disabled this test in #18598

ciyongch commented 4 years ago

The failure is observed in v1.7.x branch as well, check log http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-cpu/branches/PR-18478/runs/2/nodes/110/steps/159/log/?start=0 from PR https://github.com/apache/incubator-mxnet/pull/18478.

szha commented 4 years ago

@eric-haibin-lin actually we should have put constraint on the numpy version first before we can verify that the integration with the new version is functional, especially since numpy doesn't follow semantic versioning. Could you post a change to reflect this and to re-enable the test?

leezu commented 4 years ago

@ciyongch should this be fixed for 1.7 release? I'd expect that users would like to use latest MXNet version with latest numpy.

ciyongch commented 4 years ago

Hi @leezu, as the latest numpy (1.19.0) was released just for a couple of days, I think it's better but not mandatory to have this fix for 1.7 release, as numpy 1.19.0 is not fully verified with MXNet yet and it's very common to pin package A to some certain version of package B in python world. BTW, we're planing to cut the rc0 in the following days and will consider whether to include the fix after rc0, what do you think?

leezu commented 4 years ago

it's very common to pin package

@ciyongch that's not true. In fact it's easy to pip install --upgrade numpy on a system and get into a broken state. Users may also install other packages requiring a new version of numpy.

BTW, we're planing to cut the rc0 in the following days and will consider whether to include the fix after rc0, what do you think?

If we consider the numpy API experimental in MXNet 1.7, it's ok but certainly not ideal.

ciyongch commented 4 years ago

Yes, it could be a broken issue unless there's a constraint for the numpy version for MXNet's installation. I'm not sure whether we can state numpy API as an experimental feature or not since it's already introduced in 1.6.0 release. If this is a mandatory fix for 1.7.0, may I know if there's anyone can help on this? @szha @leezu @sandeep-krishnamurthy Thanks.

szha commented 4 years ago

Is it feasible to make mxnet compatible with all versions above 1.17? If not then we definitely would need to make a choice on the numpy version to support. I'm asking this because numpy doesn't follow semantic versioning.

leezu commented 4 years ago

Only deprecated APIs are removed. There's no need to support the deprecated APIs in older releases. So generally this should be possible. What do you think @yzhliu

ciyongch commented 4 years ago

Hi @leezu @szha @yzhliu , is there any updates for this topic, we're pending on this to drop the rc tag for 1.7 release. Based on the previous discussion, seems there're two options to go,

  1. Make MXNet compatible with all numpy versions or versions above 1.17.
  2. Add a constraint to numpy version that MXNet supports.

Or is it necessary to raise it on dev@ ? Thanks!

ciyongch commented 4 years ago

As discussed here, we'll mark numpy operator as experimental in v1.7 release and probably will move forward by taking this as an known issue (some cases are broken with latest numpy version 1.19.0). Please let me know if you have any further concerns or suggestions!