Open Evelyn-coder opened 4 years ago
panzheyi
commented
4 years ago It seems that your mxnet version can only create an array with 2^32 bits of memory, causing the overflow. The backend of mxnet often changes, so unexpected errors sometimes raised. I suggest you to use the versions of packages specified in requirements.txt.
Evelyn-coder
commented
4 years ago It seems that your mxnet version can only create an array with 2^32 bits of memory, causing the overflow. The backend of mxnet often changes, so unexpected errors sometimes raised. I suggest you to use the versions of packages specified in requirements.txt.
Thank you very much. You are really nice. Finally I solved it by installing the MXnet with right gpu version. Thanks a lot.
wanzhixiao
commented
3 years ago It seems that your mxnet version can only create an array with 2^32 bits of memory, causing the overflow. The backend of mxnet often changes, so unexpected errors sometimes raised. I suggest you to use the versions of packages specified in requirements.txt.
Thank you very much. You are really nice. Finally I solved it by installing the MXnet with right gpu version. Thanks a lot.
Hi,how do you installed mxnet-cu90 == 1.5.0b20190108? could you give a command? when i install mxnet-cu90 == 1.5.0b20190108, there is no such version
Traceback (most recent call last): File "train.py", line 172, in
main(args)
File "train.py", line 143, in main
train, eval, test, scaler = getattr(data.dataloader, dataset_setting['dataloader'])(settings)
File "/home/xinyilian/ST-MetaNet/flow-prediction/src/data/dataloader.py", line 109, in dataloader_all_sensors_seq2seq
return dataiter_all_sensors_seq2seq(train, scaler, setting), \
File "/home/xinyilian/ST-MetaNet/flow-prediction/src/data/dataloader.py", line 90, in dataiter_all_sensors_seq2seq
feature = mx.nd.array(np.stack(feature)) # [B, N, D]
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/ndarray/utils.py", line 146, in array
return _array(source_array, ctx=ctx, dtype=dtype)
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 2505, in array
arr[:] = source_array
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 449, in setitem
self._set_nd_basic_indexing(key, value)
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 715, in _set_nd_basic_indexing
self._sync_copyfrom(value)
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 881, in _sync_copyfrom
ctypes.c_size_t(source_array.size)))
File "/home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/base.py", line 253, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [19:21:13] src/ndarray/ndarray_function.cc:51: Check failed: size == to->Size() (-2000107520 vs. 2294859776) : copying size mismatch, from: 18446744065709121536 bytes, to: 9179439104 bytes.
Stack trace:
[bt] (0) /home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x4b04cb) [0x7feb9aa234cb]
[bt] (1) /home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x281c85b) [0x7feb9cd8f85b]
[bt] (2) /home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/libmxnet.so(mxnet::NDArray::SyncCopyFromCPU(void const*, unsigned long) const+0x27c) [0x7feb9cd1b59c]
[bt] (3) /home/xinyilian/.conda/envs/xyl/lib/python3.6/site-packages/mxnet/libmxnet.so(MXNDArraySyncCopyFromCPU+0x2b) [0x7feb9ca9790b]
[bt] (4) /home/xinyilian/.conda/envs/xyl/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fec1d551ec0]
[bt] (5) /home/xinyilian/.conda/envs/xyl/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7fec1d55187d]
[bt] (6) /home/xinyilian/.conda/envs/xyl/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce) [0x7fec1d767ede]
[bt] (7) /home/xinyilian/.conda/envs/xyl/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x13915) [0x7fec1d768915]
[bt] (8) python(_PyObject_FastCallDict+0x8b) [0x560bdc88ae3b]