[Test] Region prompt - Githubissues

s9roll7 / animatediff-cli-prompt-travel

animatediff prompt travel

Apache License 2.0

1.19k stars 105 forks source link

[Test] Region prompt #147

Open s9roll7 opened 11 months ago

s9roll7 commented 11 months ago

region 0 / region 1 / background
lora for region 0/1
ip adapter for each region
controlnet_openpose
region 0 / region 1 / background
lora for region 0/1
ip adapter for each region
controlnet_openpose
region 0 / region 1 / background
lora for region 0/1
ip adapter for each region
controlnet_openpose
region 0 / background
lora for region 0
ip adapter for each region
controlnet_openpose

alanhuang67 commented 11 months ago

This is so great and I'm testing for it, may I understand that we can use lora in each region but backgroynd?

s9roll7 commented 11 months ago

background is also a region The remainder after cutting out each region's mask is the background.

# new lora_map format
  "lora_map": {
        "path_to_models/models/Lora/zs_Abdiel.safetensors": {   # setting for abdiel lora
            "region" : ["0", "background"],            # target region. Multiple designations possible
            "scale" : {
                # "frame_no" : scale format
                "0": 0.75           # lora scale. same as prompt_map format. For example, it is possible to set the lora to be used from the 30th frame.
            }
        },
        "path_to_models/models/Lora/zs_Amanazoko.safetensors": {  # setting for amanozako lora
            "region" : ["1"],            # target region
            "scale" : {
                "0": 0.75
            }
        }
  },

alanhuang67 commented 11 months ago

Thanks!

if there are 2 region to control, I name region 0,background for background, and region1, region2 for other character? in this case how do I name those region in the region map section?

also one more question, if you can setup lora by region, does it possible to setup controlnet in region only? thanks!!!!!

s9roll7 commented 11 months ago

motion lora

alanhuang67 commented 11 months ago

Thanks! so my understand as below, please correct me if I'm wrong, thanks!

"lora_map": { "path_to_models/model_00" "region" : ["0", "background"], *--->this means prompt will use root setup? ... } }, "path_to_models/model_01": { # setting for amanozako lora "region" : ["1"],
... } },

"region_map" : {

I don't need to setup region 0 because it already been setup at root?

"1":{}

xxxx }

s9roll7 commented 11 months ago

"background" = root

alanhuang67 commented 11 months ago

Great! BIG THANKS!!!!

hassantsyed commented 8 months ago

do you have the configs for the first, second, or third generation here?