search

d4nst / RotNet

https://d4nst.github.io/2017/01/12/image-orientation/
MIT License
539 stars 189 forks source link

Some questions about RotNet #16

Closed dugufly closed 5 years ago

dugufly commented 5 years ago

Hello, I have a few questions to ask:

  1. In Snapseed and IPhone, there is a function to automatically detect the angle. I want to do something similar, but it's a bit different from your algorithm. In Snapseed and Iphone, the angle of return is generally between -10 degrees and 10 degrees. It usually returns 0 outside this range, but the accuracy requirement is relatively high. Pictures with water or horizons need to be corrected to the level. Pictures with characters and trees need to be corrected to vertical. How do I adjust your network?
  2. I downloaded your model, rotnet_street_view_resnet50_keras2.hdf5 This file is only 95M, and I trained more than 450M, what is the reason? Is there any way to continue compressing the model to below 20M?
  3. I need to run the program in Android. Is there any way to improve the performance? (chip SDM670, ram 4G, I expect to process a picture less than 500ms)
  4. When training, there are some values ​​in the log (loss, angle_error, var_loss, var_angle_error), what does it mean?
  5. If the training is halfway, interrupt it, how to continue training next time?

The similar features of Snapsed and IPhone are as follows: Snapseed -> open a image -> tool -> rotate IPhone -> Photos -> open a image -> Edit -> crop

d4nst commented 5 years ago
  1. You can just generate images rotated between -10 and 10 instead of between 0 and 360 by changing this line: https://github.com/d4nst/RotNet/blob/67b44e0740168f20f2b20a1fcbd01a452f00585b/utils.py#L284 You should also modify these lines to change the output of the network to 20 possible angles instead of 360: https://github.com/d4nst/RotNet/blob/67b44e0740168f20f2b20a1fcbd01a452f00585b/utils.py#L307 https://github.com/d4nst/RotNet/blob/67b44e0740168f20f2b20a1fcbd01a452f00585b/train/train_street_view.py#L27
  2. and 3. If you want to run the model on mobile, I would use something like a MobileNet, which provides fast performance and small model size.
  3. Please refer to my blog post.
  4. You can load a pre-trained model using the load_model function and then call fit_generator on this model to continue training it.
dugufly commented 5 years ago
  1. Why is angle_error always around 90, should this value converge around 0 degrees? 2 I use your model to process the pictures in the Google Street View test set, the error is about 5 degrees, how to reduce the error?
d4nst commented 5 years ago
  1. Yes, it should approach to 0 degrees. Did you change the output of the network as I mentioned in my previous post? If you do that, you shouldn't see an angle_error above 20. Also, be careful when you convert the output of your network (in your case, it will be a class between 0 and 20) and the actual rotation angle (which in your case it's between -10 and 10).
  2. There's no easy answer to that. The most straightforward way to increase the accuracy of the model is to add more training data. You could also try using a different model, modifying hyperparameters, augmenting the training set, etc.