PaddlePaddle / continuous_evaluation

Macro Continuous Evaluation Platform for Paddle.
19 stars 14 forks source link

language model core dump on CE #128

Closed Yancey1989 closed 5 years ago

Yancey1989 commented 5 years ago

job link: http://ce.paddlepaddle.org:8080/viewLog.html?buildId=1966&buildTypeId=PaddleCe_CEBuild&tab=buildLog

[05:28:57]W:     [Step 1/1] *** Aborted at 1538285337 (unix time) try "date -d @1538285337" if you are using GNU date ***
[05:28:57]W:     [Step 1/1] PC: @                0x0 (unknown)
[05:28:57]W:     [Step 1/1] *** SIGSEGV (@0x58) received by PID 7765 (TID 0x7fc7987af700) from PID 88; stack trace: ***
[05:28:57]W:     [Step 1/1]     @     0x7fc82c5a67e0 (unknown)
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8b950c PyEval_EvalFrameEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8c237d PyEval_EvalCodeEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c839905 (unknown)
[05:28:57]W:     [Step 1/1]     @     0x7fc82c807d33 PyObject_Call
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8bd0a2 PyEval_EvalFrameEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8bfe9e PyEval_EvalFrameEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8bfe9e PyEval_EvalFrameEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8c237d PyEval_EvalCodeEx
[05:28:57]W:     [Step 1/1]     @     0x7fc82c839830 (unknown)
[05:28:57]W:     [Step 1/1]     @     0x7fc82c807d33 PyObject_Call
[05:28:57]W:     [Step 1/1]     @     0x7fc82c81674d (unknown)
[05:28:57]W:     [Step 1/1]     @     0x7fc82c807d33 PyObject_Call
[05:28:57]W:     [Step 1/1]     @     0x7fc82c8b8897 PyEval_CallObjectWithKeywords
[05:28:57]W:     [Step 1/1]     @     0x7fc82c904f32 (unknown)
[05:28:57]W:     [Step 1/1]     @     0x7fc82c59eaa1 start_thread
[05:28:57]W:     [Step 1/1]     @     0x7fc82bc60bcd clone
[05:28:57]W:     [Step 1/1]     @                0x0 (unknown)
[05:28:58]W:     [Step 1/1] ./run.xsh: line 14:  7765 Segmentation fault      FLAGS_benchmark=true FLAGS_fraction_of_gpu_memory_to_use=0.0 python model.py --device=GPU --batch_size=${FLOWERS_BATCH_SIZE} --data_set=flowers --iterations=100 --gpu_id=$cudaid