Closed daniel-j-h closed 5 years ago
pytorch's adam weight decay is l2 regularization and not what we want; until it is fixed and we get proper adamw we should just ignore it. See also https://www.fast.ai/2018/07/02/adam-weight-decay/
pytorch's adam weight decay is l2 regularization and not what we want; until it is fixed and we get proper adamw we should just ignore it. See also https://www.fast.ai/2018/07/02/adam-weight-decay/