search

natanielruiz / deep-head-pose

:fire::fire: Deep Learning Head Pose Estimation using PyTorch.
Other
1.57k stars 367 forks source link

Incorrect class number + deprications #19

Closed friggog closed 5 years ago

friggog commented 6 years ago

As far as I can tell the network is trained to classify between 66 classes but there are in fact 67, bins = np.array(range(-99, 102, 3)) generates an array from -99 to 99 (inclusive) with 67 elements.

Providing the data is sanitised this is almost never an issue, but as it stands the network doesn't seem to be able handle training data between -99 and 99 degrees (inclusive), it throws an error if something in class 66 (i.e. 99 degrees) comes up as this is outside the number of classes (indexed 0-65).

Also unimportant note that transforms.Scale has been renamed to transforms.Resize in recent torch releases and nn.Softmax now required argument dim=1.

I also wonder why there is no validation during training, as it is it is hard to determine if the network is simply overfitting.

That said this is really cool work, and the results are very impressive. I've hacked together some fixes for these issues and fine-tuned the network for animal head-pose estimation with reasonable success as well.

natanielruiz commented 6 years ago

Hi @friggog, these are cool findings! I have to verify that binning thing but you may be right. Also the depricated transforms.Scale is annoying you're right, and it should be changed.

If you have fixes for all of this then you can submit a pull request :)

Cheers and thanks for the interest in this project. Very cool that you got it working on animal head pose.

friggog commented 6 years ago

I've been busy winding up the project I mentioned but should have some time now to put together a PR. I've forked and updated everything to python3/pep8 and fixed the deprecations.

Fixing the class number will of course break the pretrained models, so these would also have to be recreated and updated in the README. I don't have the datasets on hand to do this, though. I can put in a separate PR for this, and any other updates.

Also as a general point it might be interesting to look at using a MobileNet bottleneck instead of Resnet to see if performance is retained for a much smaller network. (I suspect size would be reduced from ~100mb to ~20mb).

natanielruiz commented 6 years ago

Oh yeah I forgot that models will break. Hmmmm, let me think about it for a bit and see when I have time.

I used smaller ResNet-18 and it didn't give the same results so I don't know what would happen with MobileNet bottleneck although it's interesting to try!

natanielruiz commented 6 years ago

Actually ResNet-34 gives pretty good results if anyone is interested!

Yijun88 commented 5 years ago

Hi @friggog and @natanielruiz , just wanna share a bit of using a smaller backbone. I retrained the model on EfficientNet B0 and the testing result on AFLW is slightly better then the original HopeNet (alpha=1) model. The model size was reduced to 17MB.

Thanks for the amazing work tho!