Open szha opened 4 years ago
There can also be Bus error in create_sparse_array after test_sample_multinomial_num_outputs
"Bus errors are generally caused by applications trying to access memory that hardware cannot physically address" https://stackoverflow.com/questions/36970110/how-to-determine-the-cause-for-bus-error
[2020-11-07T05:38:26.553Z] tests/python/gpu/test_operator_gpu.py::test_normal_generator PASSED [ 82%]
[2020-11-07T05:38:26.809Z] tests/python/gpu/test_operator_gpu.py::test_uniform_generator PASSED [ 83%]
[2020-11-07T05:38:30.066Z] tests/python/gpu/test_operator_gpu.py::test_gamma_generator PASSED [ 83%]
[2020-11-07T05:38:33.322Z] tests/python/gpu/test_operator_gpu.py::test_exponential_generator PASSED [ 84%]
[2020-11-07T05:38:35.831Z] tests/python/gpu/test_operator_gpu.py::test_poisson_generator PASSED [ 84%]
[2020-11-07T05:38:39.094Z] tests/python/gpu/test_operator_gpu.py::test_negative_binomial_generator PASSED [ 85%]
[2020-11-07T05:39:11.490Z] tests/python/gpu/test_operator_gpu.py::test_multinomial_generator PASSED [ 85%]
[2020-11-07T05:39:11.490Z] tests/python/gpu/test_operator_gpu.py::test_with_random_seed PASSED [ 86%]
[2020-11-07T05:39:11.490Z] tests/python/gpu/test_operator_gpu.py::test_random_seed PASSED [ 86%]
[2020-11-07T05:39:11.490Z] tests/python/gpu/test_operator_gpu.py::test_unique_zipfian_generator PASSED [ 87%]
[2020-11-07T05:39:11.490Z] tests/python/gpu/test_operator_gpu.py::test_zipfian_generator PASSED [ 87%]
[2020-11-07T05:40:47.841Z] tests/python/gpu/test_operator_gpu.py::test_shuffle PASSED [ 88%]
[2020-11-07T05:40:47.841Z] tests/python/gpu/test_operator_gpu.py::test_randint PASSED [ 88%]
[2020-11-07T05:40:47.841Z] tests/python/gpu/test_operator_gpu.py::test_randint_extremes PASSED [ 89%]
[2020-11-07T05:41:34.440Z] tests/python/gpu/test_operator_gpu.py::test_randint_generator PASSED [ 89%]
[2020-11-07T05:41:34.440Z] tests/python/gpu/test_operator_gpu.py::test_randint_without_dtype PASSED [ 90%]
[2020-11-07T05:41:34.440Z] tests/python/gpu/test_operator_gpu.py::test_sample_multinomial_num_outputs PASSED [ 90%]
[2020-11-07T05:42:00.951Z] Fatal Python error: Bus error
[2020-11-07T05:42:00.951Z]
[2020-11-07T05:42:00.951Z] Current thread 0x00007fbb4be7f740 (most recent call first):
[2020-11-07T05:42:00.951Z] File "/work/mxnet/python/mxnet/test_utils.py", line 505 in create_sparse_array
[2020-11-07T05:42:00.951Z] File "/work/mxnet/python/mxnet/test_utils.py", line 543 in create_sparse_array_zd
[2020-11-07T05:42:00.951Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 596 in check_sparse_mathematical_core
[2020-11-07T05:42:00.951Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 726 in check_binary_op_with_scalar
[2020-11-07T05:42:00.951Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 1133 in test_sparse_mathematical_core
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 167 in pytest_pyfunc_call
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 1445 in runtest
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 134 in pytest_runtest_call
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 210 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 237 in from_call
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 210 in call_runtest_hook
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 129 in call_and_report
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 99 in runtestprotocol
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 84 in pytest_runtest_protocol
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 92 in pytest_runtest_protocol
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 271 in pytest_runtestloop
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 247 in _main
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 197 in wrap_session
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 240 in pytest_cmdline_main
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/config/__init__.py", line 93 in main
[2020-11-07T05:42:00.951Z] File "/opt/rh/rh-python36/root/usr/bin/pytest", line 8 in <module>
[2020-11-07T06:01:37.785Z] Timeout (0:20:00)!
[2020-11-07T06:01:37.785Z] Thread 0x00007fbb4be7f740 (most recent call first):
[2020-11-07T06:01:37.785Z] File "/work/mxnet/python/mxnet/test_utils.py", line 505 in create_sparse_array
[2020-11-07T06:01:37.785Z] File "/work/mxnet/python/mxnet/test_utils.py", line 543 in create_sparse_array_zd
[2020-11-07T06:01:37.785Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 596 in check_sparse_mathematical_core
[2020-11-07T06:01:37.785Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 726 in check_binary_op_with_scalar
[2020-11-07T06:01:37.785Z] File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line 1133 in test_sparse_mathematical_core
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 167 in pytest_pyfunc_call
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 1445 in runtest
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 134 in pytest_runtest_call
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 210 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 237 in from_call
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 210 in call_runtest_hook
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 129 in call_and_report
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 99 in runtestprotocol
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 84 in pytest_runtest_protocol
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 92 in pytest_runtest_protocol
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 271 in pytest_runtestloop
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 247 in _main
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 197 in wrap_session
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 240 in pytest_cmdline_main
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda>
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/config/__init__.py", line 93 in main
[2020-11-07T06:01:37.785Z] File "/opt/rh/rh-python36/root/usr/bin/pytest", line 8 in <module>
[2020-11-07T08:26:36.330Z] Sending interrupt signal to process
Description
unittest/test_sparse_operator.py::test_sparse_mathematical_core segmentation fault on centos-cpu
Occurrences
https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-cpu/detail/PR-18656/18/pipeline/#step-259-log-1771