heaversm / runway-stylegan-animations

An example of how to create random landscape images from StyleGAN using Runway ML and P5.js
91 stars 16 forks source link

Some Comments #1

Open olivercoad opened 5 years ago

olivercoad commented 5 years ago

BTW good job with this, it's a great use for simplex noise. It's an elegant method of making random looping walks through latent space.

Just a few comments related to your code.

1. You no longer need data/landscape.js to set the starting latent because a is immediately overridden using the noise loops: https://github.com/heaversm/runway-stylegan-animations/blob/fd2ca7ede62baad89ac1cabcdc78f87240beff56/js/stylegan-transition.js#L28-L30

2. In the video you asked about the diameter. You're right in that it kind of sets how much randomness there is - it determines the size of the walk through the simplex noise input space. Because the loop goes round using the angle, the diameter effectively kind of determines the noise octave/frequency and how much total variation there will be.

3. The image size has been conflated with the number of dimensions in the latent vector. In StyleGAN, the latent vector has 512 dimensions. The size of the output image is independent to the latent vector and depends on the pretrained model used.

For example here are the sizes of the pretrained models from https://github.com/NVlabs/stylegan

Path Description
ffhq-1024x1024 Generated using Flickr-Faces-HQ dataset at 1024×1024.
bedrooms-256x256 Generated using LSUN Bedroom dataset at 256×256.
cars-512x384 Generated using LSUN Car dataset at 512×384.
cats-256x256 Generated using LSUN Cat dataset at 256×256.

In stylegan-transition.js, the variable imgSize should instead be called something like latentDimensions. For example here there is a NoiseLoop for each dimension of the latent vector. https://github.com/heaversm/runway-stylegan-animations/blob/fd2ca7ede62baad89ac1cabcdc78f87240beff56/js/stylegan-transition.js#L17-L19

valentinvieriu commented 4 years ago

I double what you are saying @olivercoad . Did you managed to fix the code? At this moment the code discards the initial input.