Open twinkleyang1 opened 3 weeks ago
Please check this #8
@jiangzeyinzi Thank you for your answer. I made the modification on the yaml file based on canny, but now I have encountered some problems. My data collection format Source: FILE, Target: FILE, Prompt. Source:FILE indicates the original image; Target:FILE indicates the segmentation image; Prompt: Convert to a segmentation map based on the prompt. But the result is similar to Target:FILE. I want to generate images that are similar to Source:FILE and based on the conditions of Target:FILE
Based on your description, I'm still a bit confused about which task you would like to execute: 1) Conditional generation task: Generate an image from a caption + segmentation map. 2) Editing Task A: Generate an image from a segmentation map + natural language instruction. 3) Editing Task B: Generate a segmentation map from an image + natural language instruction.
Our framework can handle all of these tasks, but the methods of use differ.
@jiangzeyinzi What I'm going to do is Conditional generation task, Generate an image from prompt+ segmentation map.
The data set format is similar to that of scepter_img2img_hed_example.
Your description is Source:FILE,Target:FILE,Prompt images/src_002.jpeg,images/tar_002.png,Convert to an edge map based on the prompt: A Seamless pattern with different cupcakes. Where your Source:FILE is the source FILE, Target:FILE is the hed edge processing file,。Your goal is to generate a file that is similar to the source file based on the hed boundary
My description is Source:FILE,Target:FILE,Prompt images/src_R2407(1).png,images/tar_R2407(1).png,Convert to a segmentation map based on the prompt: disk and cup segmentation map My Source:FILE is a source FILE, and Target:FILE is a segmentation file. My goal is to generate a file similar to the source file based on segmentation conditions
The scepter_img2img_hed_example YAML or data format you are using is supported by another work, stylebooth, for image editing tasks.
In SCEdit, we extract image conditions through on-the-fly operation by registering ANNOTATORS, so the inputs are just the prompt and the target image, as shown in the several methods/scedit/ctr/xxx.yaml
files you used earlier. The CONTROL_ANNO field in the corresponding YAML file is the operation of the condition we used.
You can achieve the required functionality by registering the ANNOTATORS of segment algo, such as Uniformer, in the modules/annotator
directory, then use it in the yaml.
Thank you for your contributions. Could you please tell me how to do segmentation-based controlled image generation