jolibrain / joliGEN

Generative AI Image Toolset with GANs and Diffusion for Real-World Applications
https://www.joligen.com
Other
238 stars 31 forks source link

The 'ref-in' parameter is indeed mandatory. #583

Closed notiqq closed 8 months ago

notiqq commented 10 months ago

Hello,

I was following guideline on how to use DDPM model on the VITON-HD dataset.

Once I have reached the following command:

mkdir -p ~/inferences
cd ~/joliGEN/scripts
python3 gen_single_image_diffusion.py \
     --model-in-file ~/checkpoints/VITON-HD/latest_net_G_A.pth \
     --img-in ~/datasets/VITON-HD/testA/imgs/00006_00.jpg \
     --mask-in ~/datasets/VITON-HD/testA/mask/00006_00.png \
     --dir-out ~/inferences \
     --nb_samples 4 \
     --img-width 256 \
     --img-height 256
 I got the following exception that ref variable is used before actually being defined. I have checked the code and the issue happens here (method called generate ) 

    if ref_in:
      ref = cv2.imread(ref_in)
      ref_orig = ref.copy()
      ref = cv2.cvtColor(ref, cv2.COLOR_BGR2RGB)

      the issue is that ref_in is None and ref is not actually gets defined. 

      I have checked the main and there is the following piece of code:
              options.parser.add_argument(
        "--ref-in", help="image used as reference", required=False
    )

It looks like it should not be required, but without passing --ref-in param into initial command it does not work properly.

beniz commented 10 months ago

Hi @notiqq of course you need to pass the reference image, see https://github.com/jolibrain/joliGEN/issues/576#issuecomment-1790656868 and https://github.com/jolibrain/joliGEN/issues/568#issuecomment-1769252307

Have you found an error in the documentation ? Let us know.

notiqq commented 10 months ago

Yes, it looks like documentation is outdated. It does not contain information that we need to pass it. image

beniz commented 10 months ago

Ah that's normal in practice, this is the tutorial on training without reference image, we may want to replace it indeed, or to improve it by adding the training/inference with reference image here as well.