Proposal of a generative model of natural 3D textures which is trained on 2D exemplars only, and provides interpolation, synthesis and reconstruction in 3D
The key inspiration is Perlin Noise revisited with NNs to match complex color relations in 3D according to the statistics of VGG activations in 2D
The approach has the best combination of similarity and diversity compared to a range of published alternatives, that are less computationally efficient
Reshaping noise to match VGG activations using MLPs can be a scalable solution to other problems in even higher dimensions, such as time, that are difficult for CNNs.
Abstract
We propose a generative model of 2D and 3D natural textures with diversity, visual fidelity and at high computational efficiency. This is enabled by a family of methods that extend ideas from classic stochastic procedural texturing (Perlin noise) to learned, deep, non-linearities. The key idea is a hard-coded, tunable and differentiable step that feeds multiple transformed random 2D or 3D fields into an MLP that can be sampled over infinite domains. Our model encodes all exemplars from a diverse set of textures without a need to be re-trained for each exemplar. Applications include texture interpolation, and learning 3D textures from 2D exemplars.
Summary
Abstract
We propose a generative model of 2D and 3D natural textures with diversity, visual fidelity and at high computational efficiency. This is enabled by a family of methods that extend ideas from classic stochastic procedural texturing (Perlin noise) to learned, deep, non-linearities. The key idea is a hard-coded, tunable and differentiable step that feeds multiple transformed random 2D or 3D fields into an MLP that can be sampled over infinite domains. Our model encodes all exemplars from a diverse set of textures without a need to be re-trained for each exemplar. Applications include texture interpolation, and learning 3D textures from 2D exemplars.
Author
Journal/Conference
Subjects
cs.CV: Computer Vision and Pattern Recognitioncs.GR: Graphicscs.LG: Machine LearningComment
Link