ndujar
commented
4 years ago Changes have been made to integrate and correct bad predictions made with the SL model. In order to facilitate re-training, the code has been refactored to separate QoE model training from tamper models training. The code for their integration in the API can be found in the qoe_model_integration branch of the repository, and it is strongly advised to put them in production only once they have been properly re-trained using real data.
The SL model used in this branch achieves a TPR of 0.98 in training with YT & Vimeo assets. However, in tests, the SL model performs poorly with a FPR of 1.0. The huge difference in performance between training and testing seems to suggest that there is an issue in how the computation is performed in the testing setting vs. the training setting (i.e. how features are scaled).
We need to debug the SL model and figure out the cause of this big difference in performance between a training and testing setting. This is a blocker for using a metamodel that combines the results of a OC-SVM model and a SL model.