Are `emb_dim` and `dim` equal by default

lucidrains / reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

MIT License

2.13k stars 256 forks source link

Are `emb_dim` and `dim` equal by default #133

Closed rookiez7 closed 3 years ago

rookiez7 commented 3 years ago

In class ReformerLM,there are 2 parameters about dim,emb_dim and 'dim'. i find you use emb_dim = default(emb_dim, dim).Is it default that emb_dim and dim are equal.i try make them diff,then erro occurs.self.norm = nn.LayerNorm(emb_dim) caused the problem i think.

lucidrains commented 3 years ago

I believe this has been fixed in the latest!