Removes adam's l2 regularization until fixed in pytorch#4429

mapbox / robosat

Semantic segmentation on aerial and satellite imagery. Extracts features such as: buildings, parking lots, roads, water, clouds

MIT License

2.02k stars 382 forks source link

Removes adam's l2 regularization until fixed in pytorch#4429 #166

Closed daniel-j-h closed 5 years ago

daniel-j-h commented 5 years ago

pytorch's adam weight decay is l2 regularization and not what we want; until it is fixed and we get proper adamw we should just ignore it. See also https://www.fast.ai/2018/07/02/adam-weight-decay/