Matte Anything!🐒

Interactive Natural Image Matting with Segment Anything Models

Authors: [Jingfeng Yao](https://github.com/JingfengYao), [Xinggang Wang](https://scholar.google.com/citations?user=qNCTLV0AAAAJ&hl=zh-CN)^:email:, [Lang Ye](https://github.com/YeL6), [Wenyu Liu](http://eic.hust.edu.cn/professor/liuwenyu/) Institute: School of EIC, HUST (^:email:) corresponding author [![arxiv paper](https://img.shields.io/badge/arxiv-paper-orange)](https://arxiv.org/abs/2306.04121) [![video](https://img.shields.io/badge/Demo-Video-blue)](https://github.com/hustvl/Matte-Anything/assets/74295796/dfe051c2-b5d1-442d-9eff-cd1fcfd1f51b) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE) [![authors](https://img.shields.io/badge/by-hustvl-green)](https://github.com/hustvl)

demo

📢 News

2024/05/04 Matte Anything has been accepted by the Journal of Image and Vision Computing!
2024/01/02 Now you can get alpha materials with MatAny! Check it now!
2023/07/01 We release a new version that enables text input and transparency correction!
2023/06/08 We release arxiv tech report!
2023/06/08 We release source codes of Matte Anything!

The program is still in progress. You can try the early version first! Thanks for your attention. If you like Matte Anything, you may also like its previous foundation work ViTMatte.

🔥 Transparent Materials of Yourself !

MatAny could get matting results as RGBA images. It will be saved into your_demos/your_demo.png. You can specify the saving name of it in our UI. We hope MatAny could help.

📜 Introduction

We propose Matte Anything (MatAny), an interactive natural image matting model. It could produce high-quality alpha-matte with various simple hints. The key insight of MatAny is to generate pseudo trimap automatically with contour and transparency prediction. We leverage task-specific vision models to enhance the performance of natural image matting.

web_ui

🌞 Features

Matte Anything with Simple Interaction
High Quality Matting Results
Ability to Process Transparent Object

🎮 Quick Start

Try our Matte Anything with our web-ui!

web_ui

Quick Installation

Install Segment Anything Models as following:

pip install git+https://github.com/facebookresearch/segment-anything.git

Install ViTMatte as following:

python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
pip install -r requirements.txt

Install GroundingDINO as following:

cd Matte-Anything
git clone https://github.com/IDEA-Research/GroundingDINO.git
cd GroundingDINO
pip install -e .

Download pretrained models SAM_vit_h, ViTMatte_vit_b, and GroundingDINO-T. Put them in ./pretrained

Run our web-ui!

python matte_anything.py

How to use

Upload the image and click on it (default: foreground point).
Click Start!.
Modify erode_kernel_size and dilate_kernel_size for a better trimap (optional).

🎬 Demo

https://github.com/hustvl/Matte-Anything/assets/74295796/dfe051c2-b5d1-442d-9eff-cd1fcfd1f51b

Visualization of SAM and MatAny on real-world data from AM-2K and P3M-500 . web_ui Visualization of SAM and MatAny on Composition-1k web_ui

📋 Todo List

[x] adjustable trimap generation
[x] arxiv tech report
[x] support user transparency correction
[x] support text input
[ ] add example data
[ ] finetune ViTMatte for better performance

🤝Acknowledgement

Our repo is built upon Segment Anything, GroundingDINO, and ViTMatte. Thanks to their work.

Citation

@article{yao2024matte,
  title={Matte anything: Interactive natural image matting with segment anything model},
  author={Yao, Jingfeng and Wang, Xinggang and Ye, Lang and Liu, Wenyu},
  journal={Image and Vision Computing},
  pages={105067},
  year={2024},
  publisher={Elsevier}
}

hustvl / Matte-Anything

readme