Lack of the deep_norm variants of transformer

lucidrains / x-transformers

A concise but complete full-attention transformer with a set of promising experimental features from various papers

MIT License

4.63k stars 395 forks source link

Lack of the deep_norm variants of transformer #198

Closed ZegangC closed 11 months ago

ZegangC commented 11 months ago

Hello, I used the "deep_norm" model with Xtransformer in the past, but after the update last week, it seems that Xtransformer no longer supports this model. Is there any intention to reintroduce it?

lucidrains commented 11 months ago

no, it will not be reintroduced