takase / share_layer_params

MIT License
28 stars 4 forks source link

Hello, I found the following problems in reproducing this paper #2

Closed zgm1ybq closed 2 years ago

zgm1ybq commented 2 years ago

The following problems were found after the first epoch progress bar appeared: encoder attn ratio: 1.0 encoder ffn ratio: 1.559270977973938 encoder attn ratio: 1.7048484086990356 encoder ffn ratio: 1.7621819972991943 encoder attn ratio: 1.8875751495361328 encoder ffn ratio: 1.9414288997650146 encoder attn ratio: 2.04665207862854 encoder ffn ratio: 2.1007089614868164 encoder attn ratio: 2.200331926345825 encoder ffn ratio: 2.264730453491211 encoder attn ratio: 2.362478017807007 encoder ffn ratio: 2.4277806282043457 decoder self attn ratio: 1.0 decoder encoder attn ratio: 1.6158223152160645 decoder ffn ratio: 1.7283568382263184 decoder self attn ratio: 1.8472503423690796 decoder encoder attn ratio: 1.9521913528442383 decoder ffn ratio: 2.0334300994873047 decoder self attn ratio: 2.1397907733917236 decoder encoder attn ratio: 2.2300188541412354 decoder ffn ratio: 2.297696828842163 decoder self attn ratio: 2.3941729068756104 decoder encoder attn ratio: 2.4793968200683594 decoder ffn ratio: 2.540433645248413 decoder self attn ratio: 2.625570058822632 decoder encoder attn ratio: 2.7085251808166504 decoder ffn ratio: 2.767578125 decoder self attn ratio: 2.8491687774658203 decoder encoder attn ratio: 2.9357192516326904 decoder ffn ratio: 2.999277114868164

Have you encountered it and how to solve it

takase commented 2 years ago

I'm not sure what problems occurred. Can you share me an error message if you have?