search

ami-iit / element_human-action-intention-recognition

8 stars 0 forks source link

Implement Guided MoE for the human action recognition and WB motion Prediction #53

Closed kouroshD closed 3 years ago

kouroshD commented 3 years ago

Following the theoretical conceptualization done in https://github.com/ami-iit/element_human-action-intention-recognition/issues/52, in this issue I will track the activity for the implementation of Guided MoE network.

kouroshD commented 3 years ago

Th implementation for Guided MoE has been done in https://github.com/ami-iit/element_human-action-intention-recognition/tree/devel/scripts/MoE. In specific, by runing this script we can train a Guided MoE model: https://github.com/ami-iit/element_human-action-intention-recognition/blob/devel/scripts/MoE/main_train_moe.py

kouroshD commented 3 years ago

For the implementation, I used the functional API (check here https://github.com/ami-iit/element_human-action-intention-recognition/blob/devel/scripts/MoE/Utilities.py) and wrote some custom layers in https://github.com/ami-iit/element_human-action-intention-recognition/blob/devel/scripts/MoE/CustomLayers.py to perform as selector part of the integration of the experts and gate outputs.

kouroshD commented 3 years ago

Here is the architecture of the model:

gate_model_trial_MoE

Here is also the results of the training:

Screenshot from 2021-09-14 18-54-48

Notice that, the loss and accuracy of the validation is better than the training set, since I have dropout layers in the gate architecture. In the experts layers, there are still some overfitting, I should see how can I reduce this overfitting. One idea is collect more and new dataset with more actions included as well.

kouroshD commented 3 years ago

One point to mention is that, in MoE case we are predicting the human action for the whole time horizon in the future, therefore it is subject tp lower accuracy wrt the case where we only recognize the next action in https://github.com/ami-iit/element_human-action-intention-recognition/issues/50

Still, there is space for further designing and tuning of the architecture and its parameters, in which I will further investigate with the new dataset that I want to collect soon.

I'll close this issue for the time being, since I have reached the goal of this issue.