Run an experiment to test one hot encoded inputs

feldberlin / wavenet

An unconditioned Wavenet implementation with fast generation.

3 stars 0 forks source link

Run an experiment to test one hot encoded inputs #2

Closed purzelrakete closed 3 years ago

purzelrakete commented 3 years ago

What

Inputs to the wavenet are normalised categorical amplitude levels. Look into using one hot encoded cateogries instead.

Why

It seems that this was done by the original authors. The performance may be better.

More info here.

Acceptance Critera

[x] Experiment shows that one-hot encoded inputs perform better

purzelrakete commented 3 years ago

Implemented. I implemented this with embeddings instead of materialising one hot inputs. The effect is substantially the same, although not exactly. With one hot vectors, you get the embeddings for two adjacent samples added together at each time step. With the embeddings layer, you get one embedding per time step. But time steps are combined by the following convolutional layers, so I would not expect this method to be worse. This implementation uses less memory.