search

ShivamShrirao / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch
https://huggingface.co/docs/diffusers
Apache License 2.0
1.89k stars 509 forks source link

Model runs but does not learn #212

Closed flixmk closed 1 year ago

flixmk commented 1 year ago

Hi

I came across a weird behavior on my machine (that is probably my own fault). Maybe you could help me out. I use the notebook (https://colab.research.google.com/github/ShivamShrirao/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb) locally without the cell to upload images. Instead i will directly give the path to my dataset in the concept_list. I am not training with any class_prompts / class_images which is why I leave these empty ("").

Apart from that I dont change anything (except paths of course).

When training on 15 images on both colab and locally i get good results on colab and nothing locally. Every run, the model generates images that do not reflect the training data at all.

Reproduction

I only changed the paths like OUTPUT_DIR. Training process, loading the model and inference works, just the model does not seem to learn anything. It keeps generating the concept it previously connected to the prompt.

Logs

No response

System Info

diffusers: 0.13.0.dev0 pytorch: 1.13.1+cu117

flixmk commented 1 year ago

solved in https://github.com/ShivamShrirao/diffusers/issues/208#issuecomment-1438118539

litaotju commented 1 year ago

It's a known issue of xformers library on 30x GPUs, try install the dev version of the xformers. This works on my 3060.

pip install xformers==0.0.17.dev466 See https://github.com/facebookresearch/xformers/issues/631