Support .safetensors format for models

[jdahlstrom]

The third-party SD ecosystem has mostly shifted to the safetensors format for distributing models, as the Python pickle format allows for arbitrary code execution on deserialization. Safetensors is safe, and easy and fast to load as well.

[bzessack]

I am trying to implement safetensors support, but my general lack of knowledge about the burn-library and the layer nomenclature in the sdxl model make it very difficult to progress. The CLIP/OpenCLIP part might already be "done". It is hard to test, because the model that I have has uses the OpenCLIP/ViT-bigG-14 model instead of the ViT-G-14 model that is referenced in the README. One of my main struggle is to get the tensor weights into the burn-nodes. I feel like I am missing something very obvious, but right now I am switching between

• implementing "from_safetensors" functions
• calling "...Config::new().init_with(...Record {})
• directly instantiating the nodes while simultaneously guessing the layer names. I tried to look at other implementations to figure out how they are handling the layer names, but my superficial searches did not yield any useful information so far.

Any insight about the burn library or the sdxl layers from the sdxl1.0 layer names (which if downloaded from https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0), will be greatly appreciated.