does the optimum-habana support sdxl controlnet image to image pipeline? - Githubissues

huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)

Apache License 2.0

142 stars 170 forks source link

does the optimum-habana support sdxl controlnet image to image pipeline? #1103

Open basantaxpatra opened 2 months ago

basantaxpatra commented 2 months ago

System Info

Working on a solution for Image-to-Image with multi controlnet conditions. Don't see any sample available.
Currently only 2 samples i see (text-to-image and text-to-video).

Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

No sample for image to image with controlnet pipeline (SDXL)

Expected behavior

Reference sample is helpful

regisss commented 1 month ago

There are some open PRs about image-to-image generation, like #869 and #988. They should be merged into main soon. Is it what you're looking for?

dsocek commented 1 month ago

@basantaxpatra FYI here is a functional SDXL + CN on Gaudi (still working on adding batching so didn't create PR yet): https://github.com/dsocek/optimum-habana/tree/sdxlcn