elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.27k stars 90 forks source link

Fix loading more recent VaeKl checkpoints #305

Closed jonatanklosko closed 6 months ago

jonatanklosko commented 6 months ago

https://github.com/huggingface/diffusers/pull/3387 changed layer names in the VaeKl implementation, so in order to load newer and older checkpoints we need to check for both layer names. In particular this is relevant to fp16 checkpoints for SDv1-4.

I changed the params mapping to support multiple layer names to try like this:

%{
  ...,
  "attentions.{m}.query" => ["attentions.{m}.to_q", "attentions.{m}.query"],
  ...
}