search

HomebrewNLP / Olmax

HomebrewNLP in JAX flavour for maintable TPU-Training
BSD 2-Clause "Simplified" License
45 stars 5 forks source link

Add configurable layer scales #57

Closed ClashLuke closed 2 years ago

ClashLuke commented 2 years ago

The code works fine, but the results are a bit difficult to read: grafik It appears like there is no real need to weigh any of these layers up or down.