search

KichangKim / DeepDanbooru

AI based multi-label girl image classification system, implemented by using TensorFlow.
MIT License
2.63k stars 260 forks source link

focal_loss #60

Closed gqy1025 closed 1 year ago

gqy1025 commented 2 years ago

Hi,

Your project is very impressive and i am trying to re-train the model used my own dataset. Since the distribution of tags is extremely unbalanced so i chose to use focal loss for this issue.

I'm curious if you have ever tried focal-loss, and what is its final loss ? and how does the model performance compared to BCE loss?

BTW, from my perspective, the learning rate of 5 is to high for focal-loss. Thank you again for bringing such a wonderful project!

KichangKim commented 2 years ago

I implemented and tested focal-loss, but it makes training speed be too slow. With same epoch (30), it still has too high loss and low F1 score. I think that training with focal loss need more iteration than normal loss.

You can test focal loss by changing training settings in project.json, like this: "loss": "focal_loss"

gqy1025 commented 2 years ago

Thanks for your reply, i am currently training resnet-v3, I would like to know what your learning rate was? still 5? I remember you mentioned earlier that the final loss was 0.006 with BCE loss.

i trained with focal_loss, and it converged really slow and the loss was around 14~15 .

KichangKim commented 2 years ago

I would like to know what your learning rate was? still 5?

Yes, it is still 5.