Additional Optimizers

[jhonnold]

RAdam - Based on: https://github.com/LiyuanLucasLiu/RAdam/blob/master/radam/radam.py#L96 AdamW - Based on: https://github.com/LiyuanLucasLiu/RAdam/blob/master/radam/radam.py#L173 Ranger - Based on: https://github.com/lessw2020/Best-Deep-Learning-Optimizers/blob/master/Ranger/ranger.py

[Luecx]

awesome :)