apache / mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

https://mxnet.apache.org

Apache License 2.0

20.75k stars 6.8k forks source link

Scala flaky test: testObjectDetectionExample and GanExampleSuite #17067

Open eric-haibin-lin opened 4 years ago

eric-haibin-lin commented 4 years ago

- testObjectDetectionExample

[ScalaTest-main-running-GanExampleSuite] INFO org.apache.mxnetexamples.gan.GanExampleSuite - GPU test only, skipped...

[ScalaTest-main-running-DiscoverySuite] INFO org.apache.mxnetexamples.profiler.ProfilerSuite - Running profiler test...

[ScalaTest-main-running-DiscoverySuite] INFO org.apache.mxnetexamples.profiler.ProfilerSuite - profile file save to /tmp

terminate called after throwing an instance of 'dmlc::Error'

GanExampleSuite:

- Example CI: Test GAN MNIST

  what():  [06:41:27] src/c_api/c_api_profile.cc:141: Check failed: !thread_profiling_data.calls_.empty(): 

Stack trace:

  [bt] (0) /work/mxnet/scala-package/native/target/libmxnet-scala.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x64) [0x7f3cbd6f7a44]

  [bt] (1) /work/mxnet/scala-package/native/target/libmxnet.so(mxnet::on_exit_api()+0x514) [0x7f3bc7355624]

  [bt] (2) /work/mxnet/scala-package/native/target/libmxnet.so(MXNDArrayFree+0x2a) [0x7f3bc73a83fa]

  [bt] (3) [0x7f3d7103f740]

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17057/1/pipeline/

ChaiBapchya commented 4 years ago

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16722/5/pipeline

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16722/6/pipeline/

Scala CPU [5], Scala MKLDNN-CPU [6]

16722

ChaiBapchya commented 4 years ago

Another one - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17482/4/pipeline

17482

ChaiBapchya commented 4 years ago

Another one - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17487/13/pipeline #17487

leezu commented 4 years ago

Again inhttp://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-17984/32/pipeline/372

cc @lanking520 @zachgk

ChaiBapchya commented 4 years ago

Again http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-18553/1/pipeline

18553