Open cgarciae opened 2 years ago
On the embedding layer, I think it'll be fine with Embed
as opposed to Embedding
primarily due to the Flax-like design, plus it seems to be the name used in especially given the first point you make on the guidelines. At the same time, I'm not opposed to Embedding
as its used in most frameworks. To the best of my knowledge of the other JAX-based NN libraries, only Trax calls it Embedding
, Haiku and Flax both call it Embed
.
@ptigwe Agreed. I'd leave it up to the implementer to decide the name.
The idea is to add more wrappers over Flax layers.
Guidelines
training
andfrozen
state, checkout the implementation of BatchNorm and Dropout as references.Layers List
Embedding
?scan
but add a flag for unrolling it.scan
but add a flag for unrolling it.If you want to work on a layer, feel free to "claim" it via a comment here so others know you are working on it.