Closed Changwei-Ouyang closed 12 months ago
Have the authors tried fine-tuning the parameters of the LayerNorm layer with it turned on? If so, what were the results?
Thanks for your interest. I believe we tried that, but I don't remember the exact results. You could easily try it by simply modifying the codes here .
Have the authors tried fine-tuning the parameters of the LayerNorm layer with it turned on? If so, what were the results?