Robust Video Matting Plugin for Foundry Nuke
Robust Video Matting (RVM) is specifically designed for robust human video matting. Unlike existing neural models that process frames as independent images, RVM uses a recurrent neural network to process videos with temporal memory. Since RVM uses RNN, it gives better result compare to modnet that uses CNN. CNNs have to process each frame separately, but an RNN can process the current frame and frames around.
Watch the video (YouTube) to see the installation, usage and model's performance.
Use the instructions bellow, or watch above video.
nuke.pluginAddPath('./KM_RVM_Nuke')
Restart nuke, now you can find RVM node via Nodes menu under "Keying" or simply search "rvm" in nodes search bar :
Download and Install MiniConda : https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
Open "Anaconda Prompt" located in "KM_RVM_Nuke" folder, or find it in windows search. Run bellow commands, line by line
conda create -n km_rvm python=3.9.0
conda activate km_rvm
pip install "av==8.0.3" "tqdm==4.61.1" "pims==0.5"
pip install torch torchvision
Replace last line with this if you want to add gpu supoort (it needs download more dependencies and huge files):
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118
, change "cu118" to your graphic card Cuda version. Cuda table is provided bellow, end of this page.
All set up, You ready to go !
If you have your own python installed and you are familiar with installing packages, install dependencies. Then in dependencies tab of the gizmo, set python path to you custom python.
In Dependencies tab, click on "Check Dependencies" button to show your system configs. Compare it to the requirements.
OS : Window
Nuke Version: 15.x, 14.x, 13.x, 12.x
create rvm node by typing KM_RVM or find it in nuke nodes menu, under keying category.
Simply connect your input, or choose file path !
If you use node input, it will make a png seq precomp first.
For feeding directly a file, you need to have it as .mp4 or png sequence.
Once job get done, roto matte node will be created. Input precomp and result will be saved in "KM_RVM_Data" folder beside your nuke project.
Chunk size : Number of frames for parallel process. Play with this number to get better result base on your hardware
## Graphic Cards CUDA Version
Use this table to find out which version of pytorch cuda you need
| Graphic Card | Compute Capability | Compatible CUDA Toolkit Version | Command
|---|---|---|---|
| NVIDIA GeForce RTX 4090 | 9.0 | 11.8 | cu118 |
| NVIDIA GeForce RTX 3080 | 7.5 | 11.8 | cu118 |
| NVIDIA GeForce RTX 3070 | 7.5 | 11.8 | cu118 |
| NVIDIA GeForce RTX 3060 | 7.5 | 11.8 | cu118 |
| NVIDIA GeForce RTX 2080 | 7.2 | 11.6 | cu116 |
| NVIDIA GeForce RTX 2070 | 7.2 | 11.6 | cu116 |
| NVIDIA GeForce GTX 1660 | 6.1 | 11.4 | cu114 |
| NVIDIA GeForce GTX 1080 | 6.1 | 11.2 | cu112 |
| NVIDIA GeForce 840M | 5.0 | 10.2 | cu102 |
## RVM Main Repo
You can access main repo from here :
[Robust Video Matting ](https://github.com/PeterL1n/RobustVideoMatting)