Recognition Optimization : Series of experiments

[FrancoisLasson]

At the moment, the TDBN is implemented using Theano, the learning phase on several labeled gestures works and a recognition function which return us the PER (% of error recognition). PER is quiet good when our model is trained on 3 or 4 gestures, however, when number of patterns increases, PER increases significantly. In fact,we're actually trying to made a TDBN learns 9 gestures, PER is equal to 61.26% on validation set and 61.52% on test set whereas, on 4 gestures PER is equals to 7.36% on validation dataset and 8.12% on test set.

My hypothesis is that huge decrease of recognition performance is due to the parameters setup, like number of epochs, learning rates, number of neurones in the CRBM's hidden layer, etc. In fact, it seems logical that our model need much more time for the learning phase if the number of gestures to learn increases. The problem is an error during RBMs learning phase is passed on CRBM and logistic regressive logistic. So, we have to work by step : RBM training, then CRBM, then Logistic regressive+CRBM.**

1) RBM parameters :

• [x] For each RBMs, use Pyplot in order to display cost evolution during training phase
• [ ] Check the impact of the learning rate on cost
• [ ] Check the influence of RBMs' hidden layers on cost
• [x] Shuffle training database before training phase
• [ ] Try to improve the normalization process (or to use positions rather than angles)
• [ ] Determine the number of epochs necessaries to stabilize the cost

2) CRBM parameters :

• [x] Use Pyplot in order to display cost evolution during pre-training phase
• [x] Determine the number of epochs necessaries to stabilize the cost
• [ ] Check the impact of the learning rate on cost
• [x] Check the influence of CRBM's hidden layer on cost
• [x] Check the influence of the number of past visible layers (memory layers) on cost

3) CRBM and Regressive Logistic layer parameters :

• [x] Use Pyplot in order to display PER evolution on validation dataset or test dataset
• [x] Determine the number of epochs necessaries to stabilize the PER
• [x] Check if the CRBM's epochs influence Logistic Regressive's epochs (Pretraining could have a bad influence on training?)
• [ ] Check the impact of the learning rate on cost
• [ ] Check the influence of number of instances for each gestures on PER
• [x] Shuffle datasets

[FrancoisLasson]

RBM optimization results :

[FrancoisLasson]

I just found one reason of this high PER. In fact, I don't shuffle dataset before the supervised learning (finetuning). For 3 or 4 gestures, this error doesn't matter but it is obvious for 9 gestures. Without parameters optimisation, shuffle RBMs, CRBMs and Logistic Regressive datasets decrease PER from 61% to : Optimization complete with best validation score of 37.609756 %, obtained at iteration 1133448, with test performance 34.605263 %

In order to understand if gestures are also a cause of this high PER, we have print the confusion matrix. Idea is to understand if gestures are recognized in the same way. Moreover, an identical posture at start in all gestures could be a reason of high PER.

Confusion matrix : [[ 223. 0. 13. 0. 0. 4. 15. 0. 0.] [ 22. 267. 5. 0. 0. 3. 2. 1. 0.] [ 41. 59. 206. 2. 0. 10. 2. 15. 4.] [ 2. 3. 16. 326. 5. 9. 41. 10. 1.] [ 1. 0. 7. 107. 343. 57. 31. 7. 14.] [ 18. 1. 22. 21. 79. 326. 37. 53. 3.] [ 9. 9. 7. 15. 45. 66. 81. 29. 8.] [ 2. 33. 11. 32. 32. 26. 77. 360. 13.] [ 4. 13. 1. 1. 46. 2. 10. 64. 518.]]

[FrancoisLasson]

TODO : In fact, the better idea to find good parametrization is : 1) Find good parameters for the CRBM+Logistic Regressive using standard parameters of RBMs, checking their influence on PER 2) In a second time, check the influence of RBMs' parameters on PER using a well configured CRBM+Logistic Regressive