balazik / ComfyUI-PuLID-Flux

PuLID-Flux ComfyUI implementation
Apache License 2.0
395 stars 28 forks source link

Using this custom node via ComfyScript - i am getting very different results than via ComfyUI editor #16

Closed lschaupp closed 1 month ago

lschaupp commented 1 month ago

Here is a minimal example:

from comfy_script.runtime.real import *
load()
from comfy_script.runtime.real.nodes import *
import comfy.model_management
from comfy.model_management import xformers_enabled, vae_dtype, get_free_memory
import uuid
import json
import io
import numpy as np
import pdb
import os
import gc
import sys
from torch._C import BoolType
import torchvision.transforms as T
from diffusers.utils import load_image
import cv2
import traceback
from math import atan2, pi, ceil
import pdb
from PIL import Image, ImageFilter
from typing import Optional
import os
import torch

transform = T.ToPILImage()

def run_pulid():
    with torch.inference_mode():
        noise = RandomNoise(52535031757376)
        model = UNETLoader('flux1-dev-fp8.safetensors', 'default')
        pulidflux = PulidFluxModelLoader('pulid_flux_v0.9.0.safetensors')
        eva_clip = PulidFluxEvaClipLoader()
        faceanalysis = PulidFluxInsightFaceLoader('CUDA')
        image, _ = LoadImage('test.jpg')
        model2 = ApplyPulidFlux(model, pulidflux, eva_clip, faceanalysis, image, 1, 0, 1, None)
        clip = DualCLIPLoader('t5xxl_fp16.safetensors', 'clip_l.safetensors', 'flux')
        clip_text_encode_positive_prompt_conditioning = CLIPTextEncode("a woman as a pirate sitting on a throne. Behind her is a ship.", clip)
        clip_text_encode_positive_prompt_conditioning = FluxGuidance(clip_text_encode_positive_prompt_conditioning, 3.5)
        guider = BasicGuider(model2, clip_text_encode_positive_prompt_conditioning)
        sampler = KSamplerSelect('euler')
        denoise = 0.95
        sigmas = BasicScheduler(model, 'beta', 20, denoise)
        image3, _ = LoadImage('scene03.png')
        vae = VAELoader('ae.safetensors')
        latent = VAEEncode(image3, vae)
        latent, _ = SamplerCustomAdvanced(noise, guider, sampler, sigmas, latent)
        image4 = VAEDecode(latent, vae)
        for idx, img in enumerate(image4):
            img = transform(img.permute(2, 0, 1))
            path = "./pulid_{idx}.jpg".format(idx=idx)
            img.save(path)
            print("Saved image as:",path)

run_pulid()

When I run the nodes in ComfyUI (visually), the pipeline works and I am receiving the correct image. However running it via ComfyScript, I am getting a complete different picture which looks nothing alike the person. Something is very off.

balazik commented 1 month ago

Sorry, but I will be glad when I find time to implement masking features. I can’t deal with another dependency that might mess up my local installation.

From looking at the code, I see you are reusing the same variable/object name (clip_text_encode_positive_prompt_conditioning):

clip_text_encode_positive_prompt_conditioning = CLIPTextEncode("a woman as a pirate sitting on a throne. Behind her is a ship.", clip)
clip_text_encode_positive_prompt_conditioning = FluxGuidance(clip_text_encode_positive_prompt_conditioning, 3.5)

Isn’t this the last reference to the object? The CLIPTextEncode object could be destroyed in the meantime. Just guessing. May be when I have some spare time I will look at this.

lschaupp commented 1 month ago

Thanks for the info. I have tried your recommendation and it did not have any effect. Just wanted to make you aware of the issue.

balazik commented 1 month ago

Ok, will try this with the new version (still in works), when I have time. Closing.