use memory efficient attention

zhvng / open-musiclm

Implementation of MusicLM, a text to music model published by Google Research, with a few modifications.

https://arxiv.org/abs/2301.11325

MIT License

511 stars 59 forks source link

Closed zhvng closed 1 year ago

zhvng commented 1 year ago

implement option to use xformers memory_efficient_attention
option to disable relative position biases to take full advantage of memory savings
option to use absolute position embeddings instead