search

apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.75k stars 6.8k forks source link

Gluon RNN cannot perform deferred initialization when specifying sequence_length parameter in the forward process #21146

Open Davidham3 opened 1 year ago

Davidham3 commented 1 year ago

Description

from mxnet import nd
from mxnet.gluon import rnn

# create a simple GRU network
net = rnn.GRU(hidden_size=1, num_layers=2, bidirectional=True, layout="NTC", use_sequence_length=True)
net.initialize()

x = nd.random.uniform(shape=(2, 3, 2))
valid_length = nd.array([1, 2])

# pass the sequence_length parameter
print(net(x, sequence_length=nd.array([1, 2])))

Error Message

---------------------------------------------------------------------------
DeferredInitializationError               Traceback (most recent call last)
File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1495, in HybridBlock.forward(self, x, *args)
   1494 try:
-> 1495     params = {k: v.data(ctx) for k, v in self._reg_params.items()}
   1496 except DeferredInitializationError:

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1495, in <dictcomp>(.0)
   1494 try:
-> 1495     params = {k: v.data(ctx) for k, v in self._reg_params.items()}
   1496 except DeferredInitializationError:

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/parameter.py:574, in Parameter.data(self, ctx)
    571     raise RuntimeError("Cannot return a copy of Parameter '%s' on ctx %s via data() " \
    572                        "because its storage type is %s. Please use row_sparse_data() " \
    573                        "instead." % (self.name, str(ctx), self._stype))
--> 574 return self._check_and_get(self._data, ctx)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/parameter.py:230, in Parameter._check_and_get(self, arr_list, ctx)
    229 if self._deferred_init:
--> 230     raise DeferredInitializationError(
    231         "Parameter '%s' has not been initialized yet because initialization was " \
    232         "deferred. Actual initialization happens during the first forward pass. " \
    233         "Please pass one batch of data through the network before accessing Parameters. " \
    234         "You can also avoid deferred initialization by specifying in_units, " \
    235         "num_features, etc., for network layers."%(self.name))
    236 raise RuntimeError(
    237     "Parameter '%s' has not been initialized. Note that " \
    238     "you should initialize parameters and create Trainer " \
    239     "with Block.collect_params() instead of Block.params " \
    240     "because the later does not include Parameters of " \
    241     "nested child Blocks"%(self.name))

DeferredInitializationError: Parameter 'gru0_l0_i2h_weight' has not been initialized yet because initialization was deferred. Actual initialization happens during the first forward pass. Please pass one batch of data through the network before accessing Parameters. You can also avoid deferred initialization by specifying in_units, num_features, etc., for network layers.

During handling of the above exception, another exception occurred:

MXNetError                                Traceback (most recent call last)
File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1192, in HybridBlock._deferred_infer_shape(self, *args)
   1191 try:
-> 1192     self.infer_shape(*args)
   1193 except Exception as e:

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1410, in HybridBlock.infer_shape(self, *args)
   1409 """Infers shape of Parameters from inputs."""
-> 1410 self._infer_attrs('infer_shape', 'shape', *args)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1394, in HybridBlock._infer_attrs(self, infer_fn, attr, *args)
   1393 """Generic infer attributes."""
-> 1394 inputs, out = self._get_graph(*args)
   1395 args, _ = _flatten(args, "input")

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1060, in HybridBlock._get_graph(self, *args)
   1059 with self.name_scope():
-> 1060     out = self.hybrid_forward(symbol, *grouped_inputs, **params)  # pylint: disable=no-value-for-parameter
   1061 out, self._out_format = _flatten(out, "output")

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/rnn/rnn_layer.py:254, in _RNNLayer.hybrid_forward(self, F, inputs, states, sequence_length, **kwargs)
    251             raise ValueError(
    252                 "Invalid recurrent state shape. Expecting %s, got %s."%(
    253                     str(info['shape']), str(state.shape)))
--> 254 out = self._forward_kernel(F, inputs, states, sequence_length, **kwargs)
    256 # out is (output, state)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/rnn/rnn_layer.py:288, in _RNNLayer._forward_kernel(self, F, inputs, states, sequence_length, **kwargs)
    287 rnn_fn = F.npx.rnn if is_np_array() else F.RNN
--> 288 rnn = rnn_fn(inputs, params, *rnn_args, use_sequence_length=self._use_sequence_length,
    289              state_size=self._hidden_size, projection_size=self._projection_size,
    290              num_layers=self._num_layers, bidirectional=self._dir == 2,
    291              p=self._dropout, state_outputs=True, mode=self._mode,
    292              lstm_state_clip_min=self._lstm_state_clip_min,
    293              lstm_state_clip_max=self._lstm_state_clip_max,
    294              lstm_state_clip_nan=self._lstm_state_clip_nan)
    296 if self._mode == 'lstm':

File <string>:199, in RNN(data, parameters, state, state_cell, sequence_length, state_size, num_layers, bidirectional, mode, p, state_outputs, projection_size, lstm_state_clip_min, lstm_state_clip_max, lstm_state_clip_nan, use_sequence_length, name, attr, out, **kwargs)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/_ctypes/symbol.py:143, in _symbol_creator(handle, args, kwargs, keys, vals, name, is_np_op, output_is_list)
    142 elif kwargs:
--> 143     s._compose(name=name, **kwargs)
    144 else:

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/symbol/symbol.py:507, in Symbol._compose(self, *args, **kwargs)
    506     args = c_handle_array(args)
--> 507 check_call(_LIB.MXSymbolCompose(
    508     self.handle, name, num_args, keys, args))

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/base.py:246, in check_call(ret)
    245 if ret != 0:
--> 246     raise get_last_ffi_error()

MXNetError: Traceback (most recent call last):
  [bt] (99) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (98) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (97) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (96) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (95) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (94) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (93) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (92) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (91) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (90) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (89) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (88) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (87) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (86) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1510) [0x555c22190eb0]
  [bt] (85) /root/miniconda3/envs/autogluon/bin/python(+0x166b2e) [0x555c2211fb2e]
  [bt] (84) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (83) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (82) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (81) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (80) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (79) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (78) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (77) /root/miniconda3/envs/autogluon/bin/python(+0x17cd15) [0x555c22135d15]
  [bt] (76) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (75) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1b5b) [0x555c221914fb]
  [bt] (74) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (73) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1b5b) [0x555c221914fb]
  [bt] (72) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (71) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (70) /root/miniconda3/envs/autogluon/bin/python(+0x13b23d) [0x555c220f423d]
  [bt] (69) /root/miniconda3/envs/autogluon/bin/python(+0x1f98df) [0x555c221b28df]
  [bt] (68) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (67) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x546a) [0x555c22194e0a]
  [bt] (66) /root/miniconda3/envs/autogluon/bin/python(_PyObject_MakeTpCall+0x3bf) [0x555c220ea13f]
  [bt] (65) /root/miniconda3/envs/autogluon/bin/python(+0x194d2b) [0x555c2214dd2b]
  [bt] (64) /root/miniconda3/envs/autogluon/bin/python(_PyObject_FastCallDict+0xe7) [0x555c221176b7]
  [bt] (63) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (62) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x888) [0x555c22185c28]
  [bt] (61) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (60) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (59) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (58) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (57) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (56) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (55) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (54) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (53) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (52) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (51) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (50) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (49) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (48) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (47) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (46) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (45) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (44) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (43) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (42) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (41) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (40) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (39) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (38) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (37) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (36) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (35) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (34) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (33) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (32) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (31) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (30) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (29) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (28) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (27) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (26) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (25) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (24) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (23) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (22) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (21) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (20) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (19) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (18) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (17) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (16) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x534) [0x555c22186b64]
  [bt] (15) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (14) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (13) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (12) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (11) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (10) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (9) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x5434) [0x555c22194dd4]
  [bt] (8) /root/miniconda3/envs/autogluon/bin/python(_PyObject_MakeTpCall+0x3bf) [0x555c220ea13f]
  [bt] (7) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(+0x13c95) [0x7fc8a64cbc95]
  [bt] (6) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(_ctypes_callproc+0x319) [0x7fc8a64cb1e9]
  [bt] (5) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/../../libffi.so.7(+0x6067) [0x7fc8a64b2067]
  [bt] (4) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/../../libffi.so.7(+0x69dd) [0x7fc8a64b29dd]
  [bt] (3) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(NNSymbolCompose+0x1c5) [0x7fc8589fa8c5]
  [bt] (2) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(nnvm::Symbol::Compose(dmlc::array_view<nnvm::Symbol const*> const&, std::unordered_map<std::string, nnvm::Symbol const*, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, nnvm::Symbol const*> > > const&, std::string const&)+0x1ba5) [0x7fc858a0f875]
  [bt] (1) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(nnvm::KeywordArgumentMismatch(char const*, std::vector<std::string, std::allocator<std::string> > const&, dmlc::array_view<std::string> const&)+0x1ff) [0x7fc858a128ff]
  [bt] (0) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4f) [0x7fc8539b309f]
[14:28:38] /work/mxnet/3rdparty/tvm/nnvm/src/core/symbolic.cc:90: Symbol.ComposeKeyword argument name state_cell not found.
Candidate arguments:
        [0]data
        [1]parameters
        [2]state
        [3]sequence_length

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Input In [1], in <cell line: 10>()
      7 x = nd.random.uniform(shape=(2, 3, 2))
      8 valid_length = nd.array([1, 2])
---> 10 print(net(x, sequence_length=nd.array([1, 2])))

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/rnn/rnn_layer.py:240, in _RNNLayer.__call__(self, inputs, states, sequence_length, **kwargs)
    237     states = [states]
    239 if self._use_sequence_length:
--> 240     return super(_RNNLayer, self).__call__(inputs, states, sequence_length, **kwargs)
    241 else:
    242     return super(_RNNLayer, self).__call__(inputs, states, **kwargs)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:825, in Block.__call__(self, *args)
    822 for hook in self._forward_pre_hooks.values():
    823     hook(self, args)
--> 825 out = self.forward(*args)
    827 for hook in self._forward_hooks.values():
    828     hook(self, args, out)

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1497, in HybridBlock.forward(self, x, *args)
   1495     params = {k: v.data(ctx) for k, v in self._reg_params.items()}
   1496 except DeferredInitializationError:
-> 1497     self._deferred_infer_shape(x, *args)
   1498     for _, v in self.params.items():
   1499         v._finish_deferred_init()

File ~/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/gluon/block.py:1196, in HybridBlock._deferred_infer_shape(self, *args)
   1193 except Exception as e:
   1194     error_msg = "Deferred initialization failed because shape"\
   1195                 " cannot be inferred. {}".format(e)
-> 1196     raise ValueError(error_msg)

ValueError: Deferred initialization failed because shape cannot be inferred. Traceback (most recent call last):
  [bt] (99) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (98) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (97) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (96) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (95) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (94) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (93) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (92) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (91) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (90) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (89) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (88) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (87) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (86) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1510) [0x555c22190eb0]
  [bt] (85) /root/miniconda3/envs/autogluon/bin/python(+0x166b2e) [0x555c2211fb2e]
  [bt] (84) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (83) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (82) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (81) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (80) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (79) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x1b7) [0x555c221867e7]
  [bt] (78) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x4c0) [0x555c2218fe60]
  [bt] (77) /root/miniconda3/envs/autogluon/bin/python(+0x17cd15) [0x555c22135d15]
  [bt] (76) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (75) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1b5b) [0x555c221914fb]
  [bt] (74) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (73) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1b5b) [0x555c221914fb]
  [bt] (72) /root/miniconda3/envs/autogluon/bin/python(+0x1955c2) [0x555c2214e5c2]
  [bt] (71) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (70) /root/miniconda3/envs/autogluon/bin/python(+0x13b23d) [0x555c220f423d]
  [bt] (69) /root/miniconda3/envs/autogluon/bin/python(+0x1f98df) [0x555c221b28df]
  [bt] (68) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (67) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x546a) [0x555c22194e0a]
  [bt] (66) /root/miniconda3/envs/autogluon/bin/python(_PyObject_MakeTpCall+0x3bf) [0x555c220ea13f]
  [bt] (65) /root/miniconda3/envs/autogluon/bin/python(+0x194d2b) [0x555c2214dd2b]
  [bt] (64) /root/miniconda3/envs/autogluon/bin/python(_PyObject_FastCallDict+0xe7) [0x555c221176b7]
  [bt] (63) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (62) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x888) [0x555c22185c28]
  [bt] (61) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (60) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (59) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (58) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (57) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (56) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (55) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (54) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (53) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (52) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (51) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (50) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (49) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (48) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (47) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (46) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (45) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (44) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (43) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (42) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (41) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (40) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (39) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (38) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (37) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (36) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (35) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x7d) [0x555c220f020d]
  [bt] (34) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (33) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (32) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (31) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (30) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (29) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (28) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (27) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (26) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (25) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (24) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (23) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (22) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0xd5f) [0x555c221860ff]
  [bt] (21) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (20) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (19) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (18) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (17) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x71b) [0x555c221900bb]
  [bt] (16) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x534) [0x555c22186b64]
  [bt] (15) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (14) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x1f07) [0x555c221918a7]
  [bt] (13) /root/miniconda3/envs/autogluon/bin/python(PyObject_Call+0x319) [0x555c220f04a9]
  [bt] (12) /root/miniconda3/envs/autogluon/bin/python(+0x166bf8) [0x555c2211fbf8]
  [bt] (11) /root/miniconda3/envs/autogluon/bin/python(_PyFunction_Vectorcall+0x594) [0x555c22186bc4]
  [bt] (10) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalCodeWithName+0x260) [0x555c22185600]
  [bt] (9) /root/miniconda3/envs/autogluon/bin/python(_PyEval_EvalFrameDefault+0x5434) [0x555c22194dd4]
  [bt] (8) /root/miniconda3/envs/autogluon/bin/python(_PyObject_MakeTpCall+0x3bf) [0x555c220ea13f]
  [bt] (7) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(+0x13c95) [0x7fc8a64cbc95]
  [bt] (6) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(_ctypes_callproc+0x319) [0x7fc8a64cb1e9]
  [bt] (5) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/../../libffi.so.7(+0x6067) [0x7fc8a64b2067]
  [bt] (4) /root/miniconda3/envs/autogluon/lib/python3.8/lib-dynload/../../libffi.so.7(+0x69dd) [0x7fc8a64b29dd]
  [bt] (3) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(NNSymbolCompose+0x1c5) [0x7fc8589fa8c5]
  [bt] (2) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(nnvm::Symbol::Compose(dmlc::array_view<nnvm::Symbol const*> const&, std::unordered_map<std::string, nnvm::Symbol const*, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, nnvm::Symbol const*> > > const&, std::string const&)+0x1ba5) [0x7fc858a0f875]
  [bt] (1) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(nnvm::KeywordArgumentMismatch(char const*, std::vector<std::string, std::allocator<std::string> > const&, dmlc::array_view<std::string> const&)+0x1ff) [0x7fc858a128ff]
  [bt] (0) /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4f) [0x7fc8539b309f]
[14:28:38] /work/mxnet/3rdparty/tvm/nnvm/src/core/symbolic.cc:90: Symbol.ComposeKeyword argument name state_cell not found.
Candidate arguments:
        [0]data
        [1]parameters
        [2]state
        [3]sequence_length

To Reproduce

just run the code in the first section

What have you tried to solve it?

from mxnet import nd
from mxnet.gluon import rnn

# create a simple GRU network
net = rnn.GRU(hidden_size=1, num_layers=2, bidirectional=True, layout="NTC", use_sequence_length=True)
net.initialize()

x = nd.random.uniform(shape=(2, 3, 2))
valid_length = nd.array([1, 2])

# try to perform deferred initialization without the sequence_length parameter, that will fail of course
# because we specified use_sequence_length=True when we constructed the network
# But in this process, the deferred initialization did finish
try:
    net(x)
except Exception:
    pass

# so this step can run successfully
print(net(x, sequence_length=nd.array([1, 2])))

so another method to avoid this bug is:

from mxnet import nd
from mxnet.gluon import rnn

# create a simple GRU network, add input_size parameter to avoid deferred initialization
net = rnn.GRU(hidden_size=1, num_layers=2, bidirectional=True, layout="NTC", use_sequence_length=True, input_size=2)
net.initialize()

x = nd.random.uniform(shape=(2, 3, 2))
valid_length = nd.array([1, 2])

print(net(x, sequence_length=nd.array([1, 2])))

Environment

We recommend using our script for collecting the diagnostic information with the following command curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python3

Environment Information mxnet 1.9.1 python 3.8 osx and ubuntu ``` # Paste the diagnose.py command output here ``` (autogluon) /mnt/workspace> curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python3----------Python Info---------- Version : 3.8.13 Compiler : GCC 7.5.0 Build : ('default', 'Mar 28 2022 11:38:47') Arch : ('64bit', 'ELF') ------------Pip Info----------- Version : 22.1.2 Directory : /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/pip ----------MXNet Info----------- Version : 1.9.1 Directory : /root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet Commit hash file "/root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source. Library : ['/root/miniconda3/envs/autogluon/lib/python3.8/site-packages/mxnet/libmxnet.so'] Build features: ✔ CUDA ✔ CUDNN ✔ NCCL ✔ CUDA_RTC ✖ TENSORRT ✔ CPU_SSE ✔ CPU_SSE2 ✔ CPU_SSE3 ✖ CPU_SSE4_1 ✖ CPU_SSE4_2 ✖ CPU_SSE4A ✖ CPU_AVX ✖ CPU_AVX2 ✔ OPENMP ✖ SSE ✖ F16C ✖ JEMALLOC ✔ BLAS_OPEN ✖ BLAS_ATLAS ✖ BLAS_MKL ✖ BLAS_APPLE ✔ LAPACK ✔ MKLDNN ✔ OPENCV ✖ CAFFE ✖ PROFILER ✔ DIST_KVSTORE ✖ CXX14 ✖ INT64_TENSOR_SIZE ✔ SIGNAL_HANDLER ✖ DEBUG ✖ TVM_OP ----------System Info---------- Platform : Linux-4.19.91-009.ali4000.alios7.x86_64-x86_64-with-glibc2.17 system : Linux node : dsw37054-6dbb9d6d5b-5tjcl release : 4.19.91-009.ali4000.alios7.x86_64 version : #1 SMP Mon Jan 25 10:47:38 CST 2021 ----------Hardware Info---------- machine : x86_64 processor : x86_64 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 15,17,19,21,23,25,27 Off-line CPU(s) list: 0-14,16,18,20,22,24,26,28-63 Thread(s) per core: 0 Core(s) per socket: 32 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz Stepping: 4 CPU MHz: 2499.996 BogoMIPS: 4999.99 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 1024K L3 cache: 33792K NUMA node0 CPU(s): 0-63 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 arat ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0033 sec, LOAD: 1.0720 sec. Error open Gluon Tutorial(en): http://gluon.mxnet.io, HTTP Error 404: Not Found, DNS finished in 0.4433908462524414 sec. Error open Gluon Tutorial(cn): https://zh.gluon.ai, , DNS finished in 0.7784233093261719 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.2974 sec, LOAD: 0.6419 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0341 sec, LOAD: 3.1754 sec. Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.5267374515533447 sec. ----------Environment---------- KMP_DUPLICATE_LIB_OK="True" KMP_INIT_AT_FORK="FALSE"
github-actions[bot] commented 1 year ago

Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue. Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly. If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on contributing to MXNet and our development guides wiki.