dykuang / RealTime-Neural-Style-Transfer-with-TF-Hub

Apache License 2.0
1 stars 1 forks source link

RealTime-Neural-Style-Transfer-with-TF-Hub

The repo creates a simple Tkinter GUI for Neural style transfer task. The core of the code is by using this Tensorflow Hub Model based on

@article{ghiasi2017exploring,
  title={Exploring the structure of a real-time, arbitrary neural artistic stylization network},
  author={Ghiasi, Golnaz and Lee, Honglak and Kudlur, Manjunath and Dumoulin, Vincent and Shlens, Jonathon},
  journal={arXiv preprint arXiv:1705.06830},
  year={2017}
}

The repo here only does forward prediction and there is no script for training or finetuning. For the latter, you will find tons of good resources online such as here or here.

The model can create stylized images/videos from your webcam, saved video or images. Though the code can run with a cpu, it is recommended that you run with a GPU, especially for the realtime rendering with webcam input. You may also use virtual camera tools to feed the rendered video from your webcam to your online meeting. For setups of these virtual camera tools on different platform, please refer to the relevant steps on this cool project: Avatarify.

Usage

Some sample style images are included in ./style_images. If you want to use some other styles, just delete the default path and then specify the path to the image whose style you would like to transfer in the GUI entry. In a terminal, run python NST_GUI.py to launch the GUI.

Image examples

Web cam examples

Video examples here are done with cpu, so fps is low. A GPU will help get a true real time result.

Feel free to expand and develop upon this repo and have fun.