does each model only contain an individual motion or can multiple types of motion be trained inside a single model?

[Njasa2k]

does the model only contain an individual motion or can you train multiple types of motion inside a single model?

Can you have a swinging a golfball motion and a riding a horse motion inside a single MotionDirector model?

[ruizhaocv]

Thanks for your interesting question. We train one MotionDirector for one motion concept. Learning multiple highly different motion concepts is more like training a general foundation model, rather than customizing it.

[TruthSearcher]

@ruizhaocv is it possible to merge different motion director models together? What if I need to generate a video with multi-text conditioning but the model only is capable of a single motion.

[ruizhaocv]

Thanks for your question. It is quite interesting. We are also considering merging different motions. Yes, it is possible, but it depends on how different the motions are. If the motions differ from each other too much, it is hard to generate reasonable results. We are still trying this and will release some example results later. Stay tuned!

If you have any other interesting ideas, feel free to open an issue. We will try them.