vislearn / ControlNet-XS

Apache License 2.0
411 stars 12 forks source link

About model size and human pose #14

Open CS123n opened 8 months ago

CS123n commented 8 months ago

Thanks for your wonderful job. I want to ask two questiones:

  1. If I want a larger controlnet-xs, do I only need to change the contol_model_ratio configuration?
  2. Does your pipeline work well for human poses?
Sipirius commented 8 months ago

1: With the current version, the only thing you need to change is indeed the control_model_ratio 2: We did not explicitly train on human poses, but there is no conceptual difference, so it should work just as well

cheers

CS123n commented 8 months ago

Thanks for your feedback! I have two more questiones:

  1. Is learning embeddings important? It causes the initialization state to be incorrect.
  2. What amount of data and number of steps are needed to train ControlNet-xs?