panzheyi
commented
4 years ago The problem was caused by DGL library. It seems that the initialization of DGL is not compatible with your MXNet. As the backend of MXNet and DGL changes (e.g., API and implementation of some basic functions), many weird problems could be encountered when using different versions.
I suggest using the same version of MXNet and DGL as I specified in requirements.txt. It has been successfully tested on many GPU servers. Otherwise, you need to change the usage of DGL/MXNet library according to the newest DGL/MXNet APIs, and it may need large amount of works.
Sean0719
Hi,
When i run this code I got the following error:
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/train.py", line 179, in
main(args)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/train.py", line 157, in main
metrics = [MAE(scaler), RMSE(scaler), IndexMAE(scaler, [0,1,2]), IndexRMSE(scaler, [0,1,2])],
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/train.py", line 79, in fit
self.process_data(epoch, train, metrics)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/train.py", line 60, in process_data
outputs = [self.net(x, is_training) for x in zip(inputs)]
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/train.py", line 60, in
outputs = [self.net(x, is_training) for x in zip(inputs)]
File "/opt/mxnet/python/mxnet/gluon/block.py", line 360, in call
return self.forward(args)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/model/seq2seq.py", line 233, in forward
states = self.encoder(feature, data)
File "/opt/mxnet/python/mxnet/gluon/block.py", line 360, in call
return self.forward(args)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/model/seq2seq.py", line 52, in forward
_data = _data + g(data, feature)
File "/opt/mxnet/python/mxnet/gluon/block.py", line 360, in call
return self.forward(*args)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/model/graph.py", line 71, in forward
g = self.get_graph_on_ctx(state.context)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/model/graph.py", line 67, in get_graph_on_ctx
self.build_graph_on_ctx(ctx)
File "/user/cs.aau.dk/seany/SourceCode/traffic-prediction/src/model/graph.py", line 57, in build_graph_on_ctx
g = DGLGraph()
File "/usr/local/lib/python3.5/dist-packages/dgl/graph.py", line 907, in init
self._msg_index = utils.zero_index(size=self.number_of_edges())
File "/usr/local/lib/python3.5/dist-packages/dgl/utils.py", line 249, in zero_index
return Index(F.zeros((size,), dtype=F.int64, ctx=F.cpu()))
File "/usr/local/lib/python3.5/dist-packages/dgl/backend/mxnet/tensor.py", line 151, in zeros
return nd.zeros(shape, dtype=dtype, ctx=ctx)
File "/opt/mxnet/python/mxnet/ndarray/utils.py", line 67, in zeros
return _zeros_ndarray(shape, ctx, dtype, kwargs)
File "/opt/mxnet/python/mxnet/ndarray/ndarray.py", line 3387, in zeros
return _internal._zeros(shape=shape, ctx=ctx, dtype=dtype, kwargs)
File "", line 34, in _zeros
File "/opt/mxnet/python/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
ctypes.byref(out_stypes)))
File "/opt/mxnet/python/mxnet/base.py", line 148, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Invalid Input: 'int64', valid values are: {'float16', 'float32', 'float64', 'int32', 'uint8'}, in operator _zeros(name="", dtype="int64", ctx="cpu(0)", shape="(0,)")
srun: error: nv-ai-03.srv.aau.dk: task 0: Exited with exit code 1
So could you please give me some suggestions? Thanks in advance.