search

ZurichNLP / sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet
Apache License 2.0
0 stars 2 forks source link

Issue with GPU vs CPu context #1

Closed bricksdont closed 6 years ago

bricksdont commented 6 years ago

Command:

python3 -m sockeye.score --source /home/user/arios/wmt17_test/data/dev.newstest2015.bpe.de --target /home/user/arios/wmt17_test/data/dev.newstest2015.bpe.en --models /mnt/storage/walle/users/mmueller/sockeye-test/model_transformer_tiny

Output:

[ERROR:__main__] Uncaught exception
Traceback (most recent call last):
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/sockeye/score.py", line 425, in <module>
    main()
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/sockeye/score.py", line 414, in main
    result = model.score(data_iter, mapid, args.batch_size, args.no_bucketing)
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/sockeye/score.py", line 208, in score
    scores = mx.nd.pick(sample_probs, labels)
  File "<string>", line 76, in pick
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/base.py", line 146, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [11:39:38] src/imperative/./imperative_utils.h:55: Check failed: inputs[i]->ctx().dev_mask() == ctx.dev_mask() (1 vs. 2) Operator pick require all inputs live on the same context. But the first argument is on gpu(0) while the 2-th argument is on cpu(0)

Stack trace returned 10 entries:
[bt] (0) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2ab998) [0x7f6e82907998]
[bt] (1) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2abda8) [0x7f6e82907da8]
[bt] (2) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x24b4bac) [0x7f6e84b10bac]
[bt] (3) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x24c371e) [0x7f6e84b1f71e]
[bt] (4) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x240559b) [0x7f6e84a6159b]
[bt] (5) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/site-packages/mxnet/libmxnet.so(MXImperativeInvokeEx+0x63) [0x7f6e84a61b03]
[bt] (6) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(ffi_call_unix64+0x4c) [0x7f6ea79461bc]
[bt] (7) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(ffi_call+0x2df) [0x7f6ea794547f]
[bt] (8) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x292) [0x7f6ea793d1d2]
[bt] (9) /home/user/mmueller/.pythonz/pythons/CPython-3.6.0/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x963b) [0x7f6ea793463b]

This is not a problem if the option --use-cpu is used.

bricksdont commented 6 years ago

Fixed by 289fb81d198820d247641fa2623ca5d4ad17214f, closing.