Enable key-value caching when there is a mask

Oufattole / meds-torch

MIT License

16 stars 2 forks source link

Enable key-value caching when there is a mask #128

Open Oufattole opened 2 weeks ago

Oufattole commented 2 weeks ago

Currently it is not allowed as the x-transformers library does not support it. See this line where x-transformers throws an error if you use a mask during generation.