search

facebookresearch / VMZ

VMZ: Model Zoo for Video Modeling
Apache License 2.0
1.04k stars 155 forks source link

What could be the reason of the decreasing training accuracy? #16

Closed think-high closed 6 years ago

think-high commented 6 years ago

Hey,

I finetuned the pretrained Sports-1m R2Plus1D model on a dataset. And below is how the training and testing accuracy looks like. (Taken from the log that the tools/train_net.py generates on completion)

train-test accuracy

Can you help me understand what could be the reason for the training accuracy to decrease? Logically training accuracy should never decrease with increasing epochs, right?

patrickpoirson commented 6 years ago

From reading the code it seems like the average training accuracy is not logged but rather the accuracy for the most recent mini batch. Check lines 151-169 in train_net.py

think-high commented 6 years ago

Oh. I see. That makes sense. Thanks @patrickpoirson. But won't it be a better idea to get the overall training accuracy in the log instead? @dutran

dutran commented 6 years ago

it is a design choice, if you choose to do a full evaluation, your training will be slow down.

think-high commented 6 years ago

Makes sense. Cool, Thanks.

think-high commented 6 years ago

@patrickpoirson, @dutran : What about the test accuracy that is written to the log file? That's for the entire testing dataset, right? PS: I am going to check the code to figure it out, but I just wanted to put the comment still before this issue is closed by someone. I don't want to open another issue for this. [In case I can't figure it out from the code.]

Thanks, Rahul Bhojwani

dutran commented 6 years ago

@think-high it is only run 100 iterations (https://github.com/facebookresearch/R2Plus1D/blob/master/tools/train_net.py#L147).