segmented upscale - Githubissues

aduchon commented 8 months ago

It seems like this should be possible, but I'm not sure what the config would look like.

My problem is that I have a close-up image with a man and a dog next to each other. When I tile-upscale the image (using RealisticVision) I end up with with either a man-faced dog, or a dog-faced man, usually the latter. I've tried all kinds of prompts to get the dog to upscale to a dog and the man to a man, but nothing works. I've found this also with a closeup of a man and woman turning into two women. The other parameters (scheduler, steps, strength, guidance) work well for the rest of the images (those that have just one subject).

Could "Segment Anything Model" determine the subject in each region and appropriately upscale it?

Or could region masks be automatically made (e.g., left-right split)? Some other solution?

s9roll7 commented 8 months ago

https://github.com/s9roll7/animatediff-cli-prompt-travel#video-stylization-with-region

# in prompt.json (generated in [1])
# [2] write the object you want to mask
# ex.) If you want to mask a person
    "stylize_config": {
        "create_mask": [
            "person","dog"
        ],
        "composite": {

aduchon commented 8 months ago

Thanks. I'll try it out. Should this work in the upscale_config as well?

s9roll7 commented 8 months ago

no tile-upscale command is completely independent of animatediff

aduchon commented 8 months ago

I was afraid of that. the tile-upscale creates great pictures for me, it just gets confused with multiple subjects near each other. any suggestions?

s9roll7 / animatediff-cli-prompt-travel

segmented upscale #168