search

OpenPOWER-BigData / HDP-tez

Apache License 2.0
1 stars 1 forks source link

Tez-dag: TestVertexImpl fails with random test cases intermittently #1

Open yussufsh opened 8 years ago

yussufsh commented 8 years ago

Following test cases fails on Power without any defined sequence.

TestVertexImpl.testVertexWithOneToOneSplitWhileRunning:3842->initAllVertices:2401 expected: but was: TestVertexImpl.testVertexTaskAttemptOutputFailure:3322 expected: but was: TestVertexImpl.testVertexVMErrorReport:3922->initAllVertices:2401 expected: but was: TestVertexImpl.testInputInitializerEventsMultipleSources:4190->startVertex:2413->startVertex:2426 expected: but was: TestVertexImpl.testInputInitializerEventsAtNew:4393->startVertex:2413->startVertex:2431 expected: but was: TestVertexImpl.testVertexWithOneToOneSplit:3780 expected: but was:

**Failure details:**
estVertexWithOneToOneSplitWhileRunning(org.apache.tez.dag.app.dag.impl.TestVertexImpl)  Time elapsed: 0.05 sec  <<< FAILURE!
java.lang.AssertionError: expected:<INITED> but was:<NEW>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:743)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at org.apache.tez.dag.app.dag.impl.TestVertexImpl.initAllVertices(TestVertexImpl.java:2401)
        at org.apache.tez.dag.app.dag.impl.TestVertexImpl.testVertexWithOneToOneSplitWhileRunning(TestVertexImpl.java:3842)

testVertexTaskAttemptOutputFailure(org.apache.tez.dag.app.dag.impl.TestVertexImpl)  Time elapsed: 0.043 sec  <<< FAILURE!
java.lang.AssertionError: expected:<OUTPUT_WRITE_ERROR> but was:<UNKNOWN_ERROR>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:743)
        at org.junit.Assert.assertEquals(Assert.java:118)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at org.apache.tez.dag.app.dag.impl.TestVertexImpl.testVertexTaskAttemptOutputFailure(TestVertexImpl.java:3322)
yussufsh commented 8 years ago

Found more log for the failure in maven output log file.

2016-07-18 12:43:14,359 ERROR [AsyncDispatcher event handler] impl.VertexImpl (VertexImpl.java:handle(1879)) - Uncaught Exception when handling event V_INIT on vertex vertex1 with vertexId vertex_100_0001_1_01 at current state NEW
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.InterruptedException
        at org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:256)
        at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3309)
        at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3286)
        at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
        at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1864)
        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:201)
        at org.apache.tez.dag.app.dag.impl.TestVertexImpl$VertexEventDispatcher.handle(TestVertexImpl.java:404)
        at org.apache.tez.dag.app.dag.impl.TestVertexImpl$VertexEventDispatcher.handle(TestVertexImpl.java:398)
        at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
        at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:109)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
        at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
        at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
        at org.apache.hadoop.yarn.event.AsyncDispatcher$GenericEventHandler.handle(AsyncDispatcher.java:248)
        ... 14 more
yussufsh commented 8 years ago

Reference JIRAs https://issues.apache.org/jira/browse/TEZ-2852 https://issues.apache.org/jira/browse/TEZ-2987