apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
https://mxnet.apache.org
Apache License 2.0
20.75k stars 6.8k forks source link

flaky test: test_gluon_data_vision.test_random_transforms #17369

Open haojin2 opened 4 years ago

haojin2 commented 4 years ago

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17358/2/pipeline

======================================================================

ERROR: test_gluon_data_vision.test_random_transforms

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/work/mxnet/tests/python/unittest/common.py", line 221, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/unittest/test_gluon_data_vision.py", line 246, in test_random_transforms

    assert_almost_equal(num_apply/float(iteration), 0.5, 0.1)

  File "/work/mxnet/python/mxnet/test_utils.py", line 602, in assert_almost_equal

    print('\n*** Maximum errors for vector of size {}:  rtol={}, atol={}\n'.format(a.size, rtol, atol))

AttributeError: 'float' object has no attribute 'size'

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=480703277 to reproduce.

--------------------- >> end captured logging << ---------------------
ChaiBapchya commented 4 years ago

Another one - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17487/4/pipeline

17487

ChaiBapchya commented 4 years ago

Another one unrelated PR - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-gpu/detail/PR-16253/6/pipeline

16253

ChaiBapchya commented 4 years ago

https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-cpu/detail/PR-18825/2/pipeline

18825

szha commented 4 years ago

https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-cpu/branches/PR-18924/runs/1/nodes/175/steps/250/log/?start=0

szha commented 4 years ago

https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-cpu/branches/PR-18965/runs/1/nodes/176/steps/247/log/?start=0