Don't do weight decay on LayerNorm

fastai / fastai2

Temporary home for fastai v2 while it's being developed

https://dev.fast.ai

Apache License 2.0

645 stars 235 forks source link

Closed richarddwang closed 4 years ago

richarddwang commented 4 years ago

It's a common practice and is also the default for Trainer of HuggingFace/transformers.

review-notebook-app[bot] commented 4 years ago

Check out this pull request on

Review Jupyter notebook visual diffs & provide feedback on notebooks.

Powered by ReviewNB

github-actions[bot] commented 4 years ago

👋 @richarddwang! We detected notebooks that are not stripped or that are out of sync with the library in this PR.

You need to run nbdev_install_git_hooks before submitting a pull request! For more information, see the CONTRIBUTING guide.

You can see the logs of the tests that triggered this message in run 202640432.