Can't Use the Projector with B&W Images

[cyrilzakka]

After successfully training StyleGAN 2 on a dataset of B&W images (1, 128, 128), I tried using the network for projecting real images into latent space. Unfortunately it seems like it was only coded with RGB in mind because I'm getting an error related to channels (1 vs. 3), as shown below. Unfortunately I can't find a place in the code where the 3 channels are defined. Is there any way of modifying the projector for B&W images?

Local submit - run_dir: results/00000-project-real-images
dnnlib: Running run_projector.project_real_images() on localhost...
Loading networks from "/content/stylegan2/network-snapshot-000215.pkl"...
Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... Compiling... Loading... Done.
Setting up TensorFlow plugin "upfirdn_2d.cu": Preprocessing... Compiling... Loading... Done.
Downloading http://d36zk2xti64re0.cloudfront.net/stylegan1/networks/metrics/vgg16_zhang_perceptual.pkl ... done
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 642, in set_shape
    unknown_shape)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 1 in both shapes must be equal, but are 1 and 3. Shapes are [1,1,128,128] and [?,3,?,?].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_projector.py", line 146, in <module>
    main()
  File "run_projector.py", line 141, in main
    dnnlib.submit_run(sc, func_name_map[subcmd], **kwargs)
  File "/content/stylegan2/dnnlib/submission/submit.py", line 343, in submit_run
    return farm.submit(submit_config, host_run_dir)
  File "/content/stylegan2/dnnlib/submission/internal/local.py", line 22, in submit
    return run_wrapper(submit_config)
  File "/content/stylegan2/dnnlib/submission/submit.py", line 280, in run_wrapper
    run_func_obj(**submit_config.run_func_kwargs)
  File "/content/stylegan2/run_projector.py", line 59, in project_real_images
    proj.set_network(Gs)
  File "/content/stylegan2/projector.py", line 114, in set_network
    self._dist = self._lpips.get_output_for(proc_images_expr, self._target_images_var)
  File "/content/stylegan2/dnnlib/tflib/network.py", line 221, in get_output_for
    out_expr = self._build_func(*final_inputs, **build_kwargs)
  File "<string>", line 132, in lpips_network
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py", line 645, in set_shape
    raise ValueError(str(e))
ValueError: Dimension 1 in both shapes must be equal, but are 1 and 3. Shapes are [1,1,128,128] and [?,3,?,?].

[sandhyalaxmiK]

pre-trained model vgg16_zhang_perceptual.pkl is trained for color images

[cyrilzakka]

@sandhyalaxmiK that's what I thought so I tried looking around for greyscale implementations but to no avail. Closest thing I could figure out was to sum of the weights of the first convolutional layer kernel in the pretrained VGG-16 and go from there but I have no clue what the implications of this are.

Edit 1: Will attempt this and report back. Really hoping I don't have to retrain my GAN using RGB https://github.com/RohitSaha/VGG_Imagenet_Weights_GrayScale_Images/blob/master/convert_vgg_grayscale.py

[koalahhh]

I also encountered the same problem. Is there any greyscale implementations under pytorch?