search

kijai / ComfyUI-ADMotionDirector

ComfyUI nodes for training AnimateDiff motion loras
Apache License 2.0
161 stars 14 forks source link

multiple input video training possible? #18

Open SWAPv1 opened 3 months ago

SWAPv1 commented 3 months ago

Hello. Is it possible to input more than one input video for training? like 3 videos of a person riding a bicycle to generalize the motion I want to get more variety and flexibility like the original paper. If so what would the comfyui look like?

AA-Developer commented 3 months ago

hi @SWAPv1 I think the current code supports this but the developer did not add an json example for this I do not know how to do this. It would be great if the developer @kijai added a json file for this process in the examples

kijai commented 3 months ago

You can load multiple videos with this node:

image

I've never actually trained anything with multiple videos, so I don't know more about the settings etc.

AA-Developer commented 3 months ago

hi @kijai Can you share the workflow please. It would be nice if it was like the examples you uploaded because they are very good. So it would be great if you could attach this file to increase the motion quality.

and Is it possible to enter Prompt for each video? For example, there are three videos of a person riding a bike, but I want to enter a different description for each video because it contains different information about each one. This can produce higher quality motion output.

kijai commented 3 months ago

hi @kijai Can you share the workflow please. It would be nice if it was like the examples you uploaded because they are very good. So it would be great if you could attach this file to increase the motion quality.

and Is it possible to enter Prompt for each video? For example, there are three videos of a person riding a bike, but I want to enter a different description for each video because it contains different information about each one. This can produce higher quality motion output.

Not possibly to do different prompts right now. There's no workflow to share, it's the same workflow, but you just add that node I showed and different videos to it's inputs.

SWAPv1 commented 3 months ago

You can load multiple videos with this node:

image

I've never actually trained anything with multiple videos, so I don't know more about the settings etc.

Thank you I think this is what I am looking for.

SWAPv1 commented 3 months ago

hi @kijai Can you share the workflow please. It would be nice if it was like the examples you uploaded because they are very good. So it would be great if you could attach this file to increase the motion quality.

and Is it possible to enter Prompt for each video? For example, there are three videos of a person riding a bike, but I want to enter a different description for each video because it contains different information about each one. This can produce higher quality motion output.

I am guessing you would use the existing example and just add the multi image batch as the input. For the prompt I guess you would resort to a general action and input the same type of motion but from different camera angles or subjects. I believe they have some examples in the original developers page.