Clarification on use of PRT and HDRIs

[gordon-lim]

With reference to Appendix I Experimental Setup, I would like to ask/clarify the following:

1. You mentioned the used of "second-order spherical harmonics" from HDRI Haven. I would like to ask how you were able to get them from the .hdr file format provided on HDRI Haven. And if you have provided the code to do so.

2. Based on my brief understanding of computer graphics, you have used PRT as well as the 163 "second-order spherical harmonics of indoor scene from HDRI Haven" to essentially light ("simulate light transport effects") the models from RenderPeople. Is this accurate?

3. If my understanding in 2 is accurate, what is the distribution of the seperate lighting conditions used for the 159120 training images. For example, did you use the "163 second-order spherical harmonics of indoor scene from HDRI Haven" for 163 models respectively and then PRT for the other 279 models? Or did you use use a mix of lighting conditions for a single model at different angles.

Thank you for reading. I look forward to your reply and learning from it.

[shunsukesaito]

It was a long time ago when I processed them. I dug my code base but couldn't find the code to process HDR images for this project. I must have used the following library (https://github.com/google/spherical-harmonics) for converting HDR into 2nd-order spherical harmonics and then did normalization of the coefficients to avoid saturation in the final rendered images. The results are all stored in ./env_sh.npy.

Based on my brief understanding of computer graphics, you have used PRT as well as the 163 "second-order spherical harmonics of indoor scene from HDRI Haven" to essentially light ("simulate light transport effects") the models from RenderPeople. Is this accurate?

No. PRT is intrinsic to geometry, meaning given any new light source represented by spherical harmonics, you can simulate light transport effect in real-time by taking inner product of PRT and illumination spherical harmonics. You can look into apps/render_data.py and apps/prt_util.py to get a better sense of how I use PRT and illumination spherical harmonics.

[shunsukesaito]

The fragment shader code is here https://github.com/shunsukesaito/PIFu/blob/30b428ba74bd7743a17c19fa20f6bfd39b1de057/lib/renderer/gl/data/prt.fs#L94

[gordon-lim]

Hi! I thank you sincerely for your reply. I have looked into the scripts you've mentioned but I'm still a bit confused... perhaps by the double mentioning of spherical harmonics in:

1. prt_render.py and render_data.py for the 2nd-order spherical harmonics from HDRI Haven (/env_sh.npy) shs = np.load('./env_sh.npy')

and

2. prt_util SH = getSHCoeffs(order, phi, theta)

So spherical harmonics seems to mean different things in both cases. Could you shed some light on this? Alternatively, I would also appreciate if you would refer to some other links/steps to take to study this further.

Thank you once again for your comprehensive reply and I hope to hear from you soon.

[shunsukesaito]

Good point! Indeed while they are both spherical harmonics, 1. is SH coefficients for illumination and 2. is the SH coefficients for transfer function. I attached the screenshot from the original Precomputed Radiance Transfer paper to give you the better overview of how the final color is computed. I would highly recommend checking out the paper to learn more about how it works.

[LogWell]

then did normalization of the coefficients to avoid saturation in the final rendered images

How to normalize the coefficient, thanks~