Let prediction of MSELoss be able to be bounded

fastai / fastai2

Temporary home for fastai v2 while it's being developed

https://dev.fast.ai

Apache License 2.0

645 stars 234 forks source link

Let prediction of MSELoss be able to be bounded #520

Closed richarddwang closed 4 years ago

richarddwang commented 4 years ago

For example, stsb task in GLUE benchmark predicts the sentiment of a sentence within [0., 5.]. If we don't bound the prediction, it can be out of interval like -0.098.

So I made MSELossFlat be able to set 2 arguments ceil and floor, so MSELossFlat(floor=0.0, ceil=5.0) will make prediction < 0. to 0.,and > 5. to 5. to solve the problem. (the two arguments are default to None)

I have succeed to replicate extactly STSB test score (84.6) of ELECTRA-small++ with this twitch, so it should be good.

review-notebook-app[bot] commented 4 years ago

Check out this pull request on

Review Jupyter notebook visual diffs & provide feedback on notebooks.

Powered by ReviewNB

hamelsmu commented 4 years ago

@richarddwang I patched a security vulnerability with CI so I will update your branch to include the latest changes.

github-actions[bot] commented 4 years ago

👋 @richarddwang! We detected notebooks that are not stripped or that are out of sync with the library in this PR.

You need to run nbdev_install_git_hooks before submitting a pull request! For more information, see the CONTRIBUTING guide.

You can see the logs of the tests that triggered this message in run 202639965.

jph00 commented 4 years ago

We're not adding new functionality until after release - feel free to send in this PR then, although when you do, please include documentation and tests in your PR.