search

okankop / Efficient-3DCNNs

PyTorch Implementation of "Resource Efficient 3D Convolutional Neural Networks", codes and pretrained models.
MIT License
765 stars 150 forks source link

Low accuracy on Jester dataset with pre-trained model #40

Open stefanobini opened 3 years ago

stefanobini commented 3 years ago

Hi, I downloaded you repository and the Jester dataset, then I followed the instructions contained in the "README.md" file to preprocess the dataset in order to obtain the files required by the framework and finally I ran the system in test mode both on the validation and test set with the pre-trained model ResNeXt-101 (jester_resnext_101_RGB_16_best.pth) and MobileNetv2 1.0x (jester_mobilenetv2_1.0x_RGB_16_best.pth). But the performance is completely different: around 3% on validation set and 10% on test set. So, I wondered if you can share with me how to reproduce your results.

pestrstr commented 1 year ago

Hi Stefano, have you solved the issue? I'm having the same issue too. I cannot reproduce the results of this paper. I have tested pre-trained weights on test videos and by closely looking at the output of the softmax layer, the model seems to always produce 0 as predicted label. I'm wondering if I'm testing it with wrong transforms / normalization criteria.