PuLID-Flux for ComfyUI

PuLID-Flux ComfyUI implementation (Alpha version)

:new: Version Updates

• V0.1.0: Working node with weight, start_at, end_at support (attn_mask not working)

Notes

This project was heavily inspired by cubiq/PuLID_ComfyUI. It is just a prototype that uses some convenient model hacks for the encoder section. I wanted to test the model’s quality before reimplementing it in a more formal manner. For better results I recommend the 16bit or 8bit GGUF model version of Flux1-dev (the 8e5m2 returns blurry backgrounds). In the examples directory you'll find some basic workflows.

Supported Flux models:

32bit/16bit (~22GB VRAM): model, encoder

8bit gguf (~12GB VRAM): model, encoder

8 bit FP8 e5m2 (~12GB VRAM): model, encoder

8 bit FP8 e4m3fn (~12GB VRAM): model, encoder

Clip and VAE (for all models): clip, vae

For GGUF models you will need to install ComfyUI-GGUF

Installation

• Install this repo into ComfyUI/custom_nodes

  git clone https://github.com/balazik/ComfyUI-PuLID-Flux.git

  • You need one of the mentioned Flux.1-dev models. Download the model into ComfyUI/models/unet, clip and encoder into ComfyUI/models/clip, VAE into ComfyUI/models/vae.
  • PuLID Flux pre-trained model goes in ComfyUI/models/pulid/.
  • The EVA CLIP is EVA02-CLIP-L-14-336, should be downloaded automatically (will be located in the huggingface directory).
  • facexlib dependency needs to be installed, the models are downloaded at first use.
  • Finally you need InsightFace with AntelopeV2, the unzipped models should be placed in ComfyUI/models/insightface/models/antelopev2.

Known issues

• ApplyPulidFlux doesn't work on HW with CUDA compute < v8.0, (when Flux FP8 it needs bfloat16).
• When the ApplyPulidFlux node is disconnected after first run, the Flux model is still influenced by the node.
• ApplyPulidFlux attn_mask is not working (in progress).

Credits

ComfyUI/ComfyUI - A powerful and modular stable diffusion GUI.

PuLID for Flux - tuning-free ID customization solution for FLUX.1-dev

cubiq PuLID_ComfyUI - PuLID ComfyUI native implementation (Thanks for the awesome work what you do Matteo :wink: ).