lighting estimation

[Len-Li]

Hi, thanks for sharing your great work!

I am curious that is it possible to do lighting estimation using GAN inversion as in Stylelight?

[FrozenBurning]

It is always possible to do the inversion. One naive way is that you can use the encoders of our dual codebook to derive latent representations from input images/patches. And then use that as a condition to sample the full panoramas. Another possible way would be using gradient-based optimization. You may need careful implementation of back-propagation when dealing with the quantization in codebooks.

[Len-Li]

It is always possible to do the inversion. One naive way is that you can use the encoders of our dual codebook to derive latent representations from input images/patches. And then use that as a condition to sample the full panoramas. Another possible way would be using gradient-based optimization. You may need careful implementation of back-propagation when dealing with the quantization in codebooks.

Thanks for your reply. I write a simple script to optimize a latent code, it seems that latent code does not update. Do you have any advice to deal with the quantization in codebooks?

[FrozenBurning]

You may refer to some optimization-based techniques which deal with back-propagation through quantized codebooks.