ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet

Soon Yau Cheong Armin Mustafa Andrew Gilbert

https://github.com/soon-yau/visconet/assets/19167278/ae58b7ab-fa76-4253-8a10-46656f234b20

Requirements

A suitable conda environment named control can be created and activated with:

conda env create -f environment.yaml
conda activate control

Files

All model and data files are in here. Including eval.zip containing all images used in human evaluation.

Gradio App

run python gradio_visconet.py

Citation

@inproceedings{visconet,
        author    = {Cheong, Soon Yau and Mustafa, Armin and Gilbert, Andrew},
        title     = {ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet},
        booktitle = {ECCV Workshop Proceedings}
        month     = {September},
        year      = {2024}}