Implement QKV logic in terms of einsum

jason9693 / MusicTransformer-tensorflow2.0

implementation of music transformer with tensorflow-2.0 (ICLR2019)

MIT License

353 stars 81 forks source link

Implement QKV logic in terms of einsum #14

Open r-wheeler opened 4 years ago

r-wheeler commented 4 years ago

You could reimplement the QKV / dense logic in terms of einsum for faster computation. An example layer here and the use here. This is how it is is now implemented in the tf2 version of bert / transformer.