Abort while training

[jayiind]

W1219 06:39:14.853305 35630 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:14.853649 35631 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:14.853781 35629 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:14.860672 35632 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:16.114899 35630 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:16.175866 35630 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:16.214041 35629 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase W1219 06:39:16.219384 35631 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase Thread [140650471823104] Forwarding __concat_0__, Aborted at 1513645756 (unix time) try "date -d @1513645756" if you are using GNU date

PC: @ 0x0 (unknown)

SIGFPE (@0x7febe07e49fe) received by PID 35544 (TID 0x7febbd691700) from PID 18446744073180957182; stack trace:

@     0x7fec56ff1390 (unknown)

W1219 06:39:16.236891 35632 MKLDNNBatchNormLayer.cpp:138] use_global_stats is invalid setting in training phase

@     0x7febe07e49fe paddle::MKLDNNConcatLayer::reshape()

[luotao1]

Why you use the MKLDNN*Layer? How do you train?

[jayiind]

I want to use the MKL library for training.

I use run_train.sh from examples/librispeech to train. I add use_mkldnn=True in train.py

[luotao1]

MKLDNN library is different from MKL libray. https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/mkl/mkldnn.md#overview

• when use_mkldnn=False (default), the program use MKL library
• when use_mkldnn=True, the program use MKLDNN library. But this function doesn't finised on ds2 model.

[jayiind]

oh ok thanks for the info But this function doesn't finised on ds2 model. - u mean mkldnn is not supported in DS2 model?

[luotao1]

We haven't test mkldnn on DS2 model, maybe there is soming wrong.

[jayiind]

ok