jina-ai / dalle-flow

🌊 A Human-in-the-Loop workflow for creating HD images from text
grpcs://dalle-flow.dev.jina.ai
2.83k stars 209 forks source link

Add automated novel concept training for SD to a new executor #114

Open AmericanPresidentJimmyCarter opened 1 year ago

AmericanPresidentJimmyCarter commented 1 year ago

Basically run this notebook but as an executor: https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb

The concepts could be automatically uploaded to huggingface or piped directly as docarrays to stable executors.

AmericanPresidentJimmyCarter commented 1 year ago

Turns out this is already out of date lol. Apparently the better implementation is here: https://github.com/XavierXiao/Dreambooth-Stable-Diffusion

AmericanPresidentJimmyCarter commented 1 year ago

OK -- looking into this more, the dreambooth method finetunes the actual model and so is not portable. This is fine -- lstein has a good, lower memory implementation of the training script that works with the weights from the checkpoint we use otherwise. Then we can just automate the uploading step to huggingface according to the diffusers script above.