This repo implements the DiffSeg segmentation method in the paper Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion
.
@article{tian2023diffuse,
title={Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion},
author={Tian, Junjiao and Aggarwal, Lavisha and Colaco, Andrea and Kira, Zsolt and Gonzalez-Franco, Mar},
journal={arXiv preprint arXiv:2308.12469},
year={2023}
}
DiffSeg is an unsupervised zero-shot segmentation method using attention information from a stable-diffusion model. This repo implements the main DiffSeg algorithm and additionally include an experimental feature to add semantic labels to the masks based on a generated caption.
More details can be found on the project page: https://sites.google.com/corp/view/diffseg/home
cd diffseg
conda create --name diffseg python=3.9
conda activate diffseg
pip install -r path/to/requirements.txt
Please see the instructions in the diffseg.ipynb
for running instructions.
We benchmark the performance of DiffSeg on CoCo-Stuff-27 and Cityscapes. Please see instructions in benchmarks.ipynb
.