search

ARM-software / ML-KWS-for-MCU

Keyword spotting on Arm Cortex-M Microcontrollers
Apache License 2.0
1.14k stars 417 forks source link

请教：从0开始按照说明进行训练并测试，得到的结果和给出的数据差距很大，是哪里出了问题？ #38

Closed Cxywzx closed 6 years ago

Cxywzx commented 6 years ago

你好！ 我从0开始按照说明进行训练并测试，得到的结果和给出的数据差距很大，是哪里出了问题？

下面是我的训练过程记录：

# python train.py --model_architecture ds_cnn --model_size_info 5 64 10 4 2 2 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 --dct_coefficient_count 10 --window_size_ms 40 --window_stride_ms 20 --learning_rate 0.0005,0.0001,0.00002 --how_many_training_steps 10000,10000,10000 --summaries_dir work/DS_CNN/DS_CNN1/retrain_logs --train_dir work/DS_CNN/DS_CNN1/training/
......
INFO:tensorflow:Step #29592: rate 0.000020, accuracy 93.00%, cross entropy 0.214778
INFO:tensorflow:Step #29593: rate 0.000020, accuracy 95.00%, cross entropy 0.247013
INFO:tensorflow:Step #29594: rate 0.000020, accuracy 97.00%, cross entropy 0.160600
INFO:tensorflow:Step #29595: rate 0.000020, accuracy 95.00%, cross entropy 0.125619
INFO:tensorflow:Step #29596: rate 0.000020, accuracy 95.00%, cross entropy 0.190318
INFO:tensorflow:Step #29597: rate 0.000020, accuracy 94.00%, cross entropy 0.218497
INFO:tensorflow:Step #29598: rate 0.000020, accuracy 97.00%, cross entropy 0.137457
INFO:tensorflow:Step #29599: rate 0.000020, accuracy 96.00%, cross entropy 0.194881
INFO:tensorflow:Step #29600: rate 0.000020, accuracy 95.00%, cross entropy 0.245867
INFO:tensorflow:Confusion Matrix:
 [[258   0   0   0   0   0   0   0   0   0   0   0]
 [  1 218   1   2   2   7   6   8   7   2   0   4]
 [  2   3 247   4   0   0   2   0   0   0   0   3]
 [  0   9   0 246   1   3   2   0   0   0   0   9]
 [  2   5   0   0 238   0   0   0   0  13   1   1]
 [  0   4   1  12   0 238   0   0   0   0   1   8]
 [  1   2  13   1   1   0 226   2   1   0   0   0]
 [  0   7   0   1   1   0   3 243   1   0   0   0]
 [  4   4   0   0   2   1   2   0 239   5   0   0]
 [  0   0   0   0  16   0   1   0   2 233   2   2]
 [  2   3   0   0   7   0   2   0   0   5 227   0]
 [  5   8   1   8   3   1   0   0   2   5   1 226]]
INFO:tensorflow:Step 29600: Validation accuracy = 91.79% (N=3093)
INFO:tensorflow:So far the best validation accuracy is 92.31%
INFO:tensorflow:Step #29601: rate 0.000020, accuracy 97.00%, cross entropy 0.193472
INFO:tensorflow:Step #29602: rate 0.000020, accuracy 98.00%, cross entropy 0.086266
INFO:tensorflow:Step #29603: rate 0.000020, accuracy 95.00%, cross entropy 0.206013
INFO:tensorflow:Step #29604: rate 0.000020, accuracy 95.00%, cross entropy 0.130586
INFO:tensorflow:Step #29605: rate 0.000020, accuracy 99.00%, cross entropy 0.107168
INFO:tensorflow:Step #29606: rate 0.000020, accuracy 99.00%, cross entropy 0.051911
......
INFO:tensorflow:Step #29969: rate 0.000020, accuracy 98.00%, cross entropy 0.121484
INFO:tensorflow:Step #29970: rate 0.000020, accuracy 99.00%, cross entropy 0.062734
INFO:tensorflow:Step #29971: rate 0.000020, accuracy 96.00%, cross entropy 0.131749
INFO:tensorflow:Step #29972: rate 0.000020, accuracy 96.00%, cross entropy 0.148196
INFO:tensorflow:Step #29973: rate 0.000020, accuracy 100.00%, cross entropy 0.074228
INFO:tensorflow:Step #29974: rate 0.000020, accuracy 95.00%, cross entropy 0.156531
INFO:tensorflow:Step #29975: rate 0.000020, accuracy 97.00%, cross entropy 0.098559
INFO:tensorflow:Step #29976: rate 0.000020, accuracy 95.00%, cross entropy 0.156327
INFO:tensorflow:Step #29977: rate 0.000020, accuracy 90.00%, cross entropy 0.212709
INFO:tensorflow:Step #29978: rate 0.000020, accuracy 94.00%, cross entropy 0.246338
INFO:tensorflow:Step #29979: rate 0.000020, accuracy 94.00%, cross entropy 0.208315
INFO:tensorflow:Step #29980: rate 0.000020, accuracy 95.00%, cross entropy 0.195625
INFO:tensorflow:Step #29981: rate 0.000020, accuracy 90.00%, cross entropy 0.289182
INFO:tensorflow:Step #29982: rate 0.000020, accuracy 96.00%, cross entropy 0.146916
INFO:tensorflow:Step #29983: rate 0.000020, accuracy 95.00%, cross entropy 0.153784
INFO:tensorflow:Step #29984: rate 0.000020, accuracy 95.00%, cross entropy 0.182212
INFO:tensorflow:Step #29985: rate 0.000020, accuracy 94.00%, cross entropy 0.174497
INFO:tensorflow:Step #29986: rate 0.000020, accuracy 97.00%, cross entropy 0.140645
INFO:tensorflow:Step #29987: rate 0.000020, accuracy 95.00%, cross entropy 0.160968
INFO:tensorflow:Step #29988: rate 0.000020, accuracy 98.00%, cross entropy 0.063577
INFO:tensorflow:Step #29989: rate 0.000020, accuracy 93.00%, cross entropy 0.165674
INFO:tensorflow:Step #29990: rate 0.000020, accuracy 98.00%, cross entropy 0.096141
INFO:tensorflow:Step #29991: rate 0.000020, accuracy 96.00%, cross entropy 0.149724
INFO:tensorflow:Step #29992: rate 0.000020, accuracy 92.00%, cross entropy 0.281510
INFO:tensorflow:Step #29993: rate 0.000020, accuracy 93.00%, cross entropy 0.205289
INFO:tensorflow:Step #29994: rate 0.000020, accuracy 89.00%, cross entropy 0.282349
INFO:tensorflow:Step #29995: rate 0.000020, accuracy 96.00%, cross entropy 0.124107
INFO:tensorflow:Step #29996: rate 0.000020, accuracy 97.00%, cross entropy 0.140024
INFO:tensorflow:Step #29997: rate 0.000020, accuracy 97.00%, cross entropy 0.128435
INFO:tensorflow:Step #29998: rate 0.000020, accuracy 94.00%, cross entropy 0.151532
INFO:tensorflow:Step #29999: rate 0.000020, accuracy 96.00%, cross entropy 0.143669
INFO:tensorflow:Step #30000: rate 0.000020, accuracy 93.00%, cross entropy 0.231711
INFO:tensorflow:Confusion Matrix:
 [[258   0   0   0   0   0   0   0   0   0   0   0]
 [  1 216   1   3   3   6   6   7   8   2   0   5]
 [  3   2 247   4   0   1   2   0   0   0   0   2]
 [  0   9   0 248   1   2   2   0   0   0   0   8]
 [  2   7   0   0 235   0   0   0   0  14   2   0]
 [  0   4   1  16   0 233   0   0   0   0   1   9]
 [  1   2  12   1   1   0 227   2   1   0   0   0]
 [  0   7   0   1   1   0   2 243   1   1   0   0]
 [  4   4   0   0   2   1   2   0 239   5   0   0]
 [  0   1   0   0  13   0   1   0   2 235   2   2]
 [  2   3   0   0   8   0   2   0   0   4 226   1]
 [  6   8   1  10   2   1   0   0   2   5   1 224]]
INFO:tensorflow:Step 30000: Validation accuracy = 91.53% (N=3093)
INFO:tensorflow:So far the best validation accuracy is 92.31%
INFO:tensorflow:set_size=3081
INFO:tensorflow:Confusion Matrix:
 [[257   0   0   0   0   0   0   0   0   0   0   0]
 [  0 226   2   3   0   0   3   5   3   2   5   8]
 [  0   5 239   1   0   2   7   1   0   0   0   1]
 [  0   2   1 237   3   3   2   0   0   1   0   3]
 [  0   2   0   0 248   0   3   0   3  11   3   2]
 [  1   5   0   7   2 230   1   0   0   0   0   7]
 [  0   3   4   0   2   0 256   2   0   0   0   0]
 [  1   7   1   0   1   1   2 245   0   0   1   0]
 [  0   2   0   0   2   1   1   1 238   1   0   0]
 [  0   2   0   0   6   1   1   0   8 240   2   2]
 [  0   6   1   1   5   3   0   1   0   1 228   3]
 [  0   4   1   7   1   2   3   4   1   6   0 222]]
INFO:tensorflow:Final test accuracy = 93.02% (N=3081)

# python test.py --model_architecture ds_cnn --model_size_info 5 64 10 4 2 2 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 --checkpoint work/DS_CNN/DS_CNN1/training/be/ds_cnn_9230.ckpt-12400
2018-06-07 15:49:56.929040: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2018-06-07 15:49:57.088431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties: 
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:65:00.0
totalMemory: 10.92GiB freeMemory: 10.76GiB
2018-06-07 15:49:57.088490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-07 15:49:57.311486: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-07 15:49:57.311542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-06-07 15:49:57.311552: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-06-07 15:49:57.311777: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10413 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:65:00.0, compute capability: 6.1)
2018-06-07 15:50:25.113020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-06-07 15:50:25.113082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-06-07 15:50:25.113092: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-06-07 15:50:25.113100: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-06-07 15:50:25.113310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10413 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:65:00.0, compute capability: 6.1)
INFO:tensorflow:wav_filename_placeholder: Tensor("Placeholder:0", shape=(), dtype=string)
INFO:tensorflow:Restoring parameters from work/DS_CNN/DS_CNN1/training/best/ds_cnn_9230.ckpt-12400
INFO:tensorflow:set_size=22246
INFO:tensorflow:Confusion Matrix:
 [[1827    0    0    0    0    0    0    0    0    0    0    0]
 [   0 1723    0   89    0   31    1    0    0    0    0   10]
 [   0 1509  150  186    0   36    3    0    0    0    0    1]
 [   0  635    0 1152    0   23    0    0    0    0    0   15]
 [   0 1762    0   97    0   38    2    0    0    0    0    8]
 [   0 1191    0  196    0  386    0    0    0    0    0   10]
 [   0 1572   12  187    0   19   51    0    0    0    1    3]
 [   1 1793    0   36    0   21    9   23    0    0    0    6]
 [   0 1760    1   50    0   55    0    0    2    0    0    4]
 [   0 1635    2  111    0   26    0    0    0    0    0   13]
 [   0 1774    3   95    0   79    1    0    0    0    0    5]
 [   0 1210    0  390    0   53    3    0    0    0    0  159]]
INFO:tensorflow:Training accuracy = 24.60% (N=22246)
INFO:tensorflow:set_size=3093
INFO:tensorflow:Confusion Matrix:
 [[258   0   0   0   0   0   0   0   0   0   0   0]
 [  0 240   0  12   0   4   0   0   0   0   0   2]
 [  0 200  21  33   0   7   0   0   0   0   0   0]
 [  0 109   0 156   0   5   0   0   0   0   0   0]
 [  0 243   0  11   0   3   1   0   0   0   0   2]
 [  0 178   0  38   0  40   0   0   0   0   0   8]
 [  0 222   0  15   0   2   7   0   0   0   0   1]
 [  0 248   0   3   0   1   0   1   0   0   0   3]
 [  0 243   0   9   0   4   0   0   0   0   0   1]
 [  0 242   0   9   0   3   0   0   0   0   0   2]
 [  1 228   0  13   0   4   0   0   0   0   0   0]
 [  0 175   0  56   0   5   0   0   0   0   0  24]]
INFO:tensorflow:Validation accuracy = 24.15% (N=3093)
INFO:tensorflow:set_size=3081
INFO:tensorflow:Confusion Matrix:
 [[257   0   0   0   0   0   0   0   0   0   0   0]
 [  0 240   0  10   0   6   0   0   0   0   0   1]
 [  0 193  21  31   0   9   0   0   0   0   0   2]
 [  0  84   0 158   0   8   0   0   0   0   0   2]
 [  0 244   0  14   0  12   1   0   0   0   0   1]
 [  0 147   0  40   0  64   0   0   0   0   0   2]
 [  0 220   1  33   0   3   9   0   0   0   0   1]
 [  0 234   0   9   0   7   2   3   0   0   0   4]
 [  0 221   0  12   0  13   0   0   0   0   0   0]
 [  0 223   1  26   0   8   0   0   0   0   0   4]
 [  0 203   0  17   0  28   0   0   0   0   0   1]
 [  0 153   1  67   0  12   0   0   0   0   0  18]]
INFO:tensorflow:Test accuracy = 24.99% (N=3081)
#
navsuda commented 6 years ago

Hi @Cxywzx, You are missing out "--dct_coefficient_count 10 --window_size_ms 40 --window_stride_ms 20" in your test command. python test.py --model_architecture ds_cnn --model_size_info 5 64 10 4 2 2 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 64 3 3 1 1 --checkpoint work/DS_CNN/DS_CNN1/training/be/ds_cnn_9230.ckpt-12400

navsuda commented 6 years ago

Closing the issue, please reopen the issue if you still face the issue.