We tackle the problem of texture synthesis in the setting where many input images are given and a large-scale output is required. We build on recent generative adversarial networks and propose two extensions in this paper. First, we propose an algorithm to combine outputs of GANs trained on a smaller resolution to produce a large-scale plausible texture map with virtually no boundary artifacts. Second, we propose a user interface to enable artistic control. Our quantitative and qualitative results showcase the generation of synthesized high-resolution maps consisting of up to hundreds of megapixels as a case in point.
Watch our video on Youtube:
Some of our results can be viewed interactively on EasyZoom:
The TileGAN application consists of two independent processes, the server and the client. Both can be run locally on your machine or you can choose to run the server on a remote location, depending on your hardware setup. All network operations are performed by the server process, which sends the result to the client for displaying.
.zip
to ./data
. There should be a separate folder for each dataset in ./data
(e.g. ./data/vangogh
) containing a *_network.pkl
, a *_descriptors.hdf5
, a *_clusters.hdf5
and a *_kmeans.joblib
file.requirements-pip.txt
git clone https://github.com/nmslib/hnswlib.git
cd hnswlib/python_bindings
python setup.py install
python tileGAN_server.py
conda install -c conda-forge qt pyside2
requirements-pip.txt
python tileGAN_client.py XX.XX.XX.XX
(insert the IP from the server). If you're running the server on the same machine as the application, you can omit the IP address or use 'localhost'.create_dataset path_to_pkl num_latents t_size num_clusters network_name
(expected to take between 10 and 60 minutes depending on the specified sample size)
path_to_pkl
the path to the trained network pickle of your ProGAN networknum_latents
the size of the database entries (a good number would be 50K to 300K)t_size
the size of the output descriptors (an even number somewhere around 12 and 24)num_clusters
the number of clusters (approx. 8-16)network_name
the name you want to assign your network
Anna Frühstück, Ibraheem Alhashim, Peter Wonka Contact: anna.fruehstueck (at) kaust.edu.sa
If you use this code for your research, please cite our paper:
@article{Fruehstueck2019TileGAN,
title = {{TileGAN}: Synthesis of Large-Scale Non-Homogeneous Textures},
author = {Fr\"{u}hst\"{u}ck, Anna and Alhashim, Ibraheem and Wonka, Peter},
journal = {ACM Transactions on Graphics (Proc. SIGGRAPH) },
issue_date = {July 2019},
volume = {38},
number = {4},
pages = {58:1-58:11},
year = {2019}
}
Our project is based on ProGAN. We'd like to thank Tero Karras et al. for their great work and for making their code available.