Focal loss support

[inkrement]

I just implemented prototypical focal loss support for Fasttext [1]. This loss is often used if your trainings set consists of many labels and/or is highly imbalanced [2]. For my purposes it works quite well, therefore, I wanted to ask if there is interest in greater support? If so, I could prepare a push request. However, this requires additional parameters and, in consequence, minor modifications to the used file-format.

• [1] https://github.com/inkrement/fastText
• [2] https://arxiv.org/abs/1708.02002

[dongrixinyu]

I wanna apply focal loss to fasttext. But it seems this tool does not provide the corresponding interface. Could you tell me how to apply focal loss to fasttext? Thx!

[inkrement]

@dongrixinyu Hi, good to hear that! You just have to fork/use my repository. There is already support for both the command line interface and python package! However, it is not really documented yet. Nevertheless, you can set the loss-argument to "focal" and you also have "beta" and "gamma" arguments.

[dongrixinyu]

@inkrement Hi, I find your repository is excellent! I have successfully applied focal loss to my model. However I am new to c++, and still have some questions:

1. I do not know how to tune beta and gamma. Could you provide a python example?
2. My model is overfitted and I wanna add l2 regularization or dropout. Does this repository support this?

Thx!!!

[inkrement]

@dongrixinyu both questions are not really related to the focal loss per se. Therefore, I wrote you a PM.

[dongrixinyu]

@inkrement Thank you for your reply very much! I have read your email. I find a repository https://github.com/mwydmuch/extremeText。It provides l2 and sample weight params.