search

tumurzakov / AnimateDiff

AnimationDiff with train
Apache License 2.0
111 stars 28 forks source link

lora_trainer.py seems to be expecting images #13

Open peteromallet opened 11 months ago

peteromallet commented 11 months ago

I'm trying to train with the lora_trainer. I have folders set up in the correct format with mp4 videos inside them for training

However, when I try to run it, I'm getting the below error that seems to say it's expecting images.

Any idea what's going on here? Am I supposed to be feeding in individual frames rather than videos?

File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 396, in call return self.format_row(pa_table) File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 485, in format_row formatted_batch = self.format_batch(pa_table) File "/root/miniconda3/envs/animatediff2/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 515, in format_batch return self.transform(batch) File "/workspace/AnimateDiff/train_lora.py", line 670, in preprocess_train images = [image.convert("RGB") for image in examples[image_column]] File "/workspace/AnimateDiff/train_lora.py", line 670, in images = [image.convert("RGB") for image in examples[image_column]] AttributeError: 'str' object has no attribute 'convert' Steps: 0%| | 0/700 [00:00<?, ?it/s]

tumurzakov commented 11 months ago

Hi, train lora/dreambooth with animatediff seems to be wrong way. I got pure results with it. The idea was to train multiple frames simultanously. But it led to background overfit without any generalization of training object. Try to train lora traditional way. It will work with animatediff too.