search

HomebrewNLP / Olmax

HomebrewNLP in JAX flavour for maintable TPU-Training
BSD 2-Clause "Simplified" License
45 stars 5 forks source link

SM3 instead of Adam in Adam#Shampoo #69

Closed ClashLuke closed 1 year ago

ClashLuke commented 1 year ago

Worse than baseline. SM3#Shampoo at the top, Adam#Shampoo at the bottom: grafik grafik

PR Only opened to keep track of the results