Investigate bi-directional path tracing for improved lighting resolution

[gkjohnson]

Mirrors, etc

https://www.pbr-book.org/3ed-2018/Light_Transport_III_Bidirectional_Methods/Bidirectional_Path_Tracing

[gkjohnson]

Hey @erichlof! Do you have a good simple reference on Bi Directional path tracing? Am I right in understanding that instead of directly sampling the light you cast a ray from the light to somewhere in the scene, and then directly sample that point using MIS as you traverse a random ray path? I assume you compute attenuation and PDFs along the light-ray path as well as the camera-ray path before combining them for the lighting contribution when connecting the paths.

I'm thinking that bidirectional path tracing would help address #247 as well as some more complex lighting scenarios.

[erichlof]

@gkjohnson Hi Garrett! I've been following your updates and Twitter posts - awesome stuff going on here! Congrats!

About Bi-Directional Path Tracing ('BDPT' from here on), I don't really do the full, mathematically/statistically rigorous BDPT as outlined in Chapter 10 of Eric Veach's seminal PhD thesis from 1997, 'Robust Monte Carlo Methods for Light Transport Simulation' (on which modern BDPT with MIS is based, although it was first formally mentioned and proposed 4 years earlier by Eric Lafortune in his 1993 paper simply entitled 'Bi-Directional Path Tracing').

These 2 papers are the original 'source' for the strategies and motivations for using BDPT. However, I must admit that although I can follow them for the first couple of pages, after page 3 or so, my eyes start to glaze over, ha. It is unfortunate that both authors are coming up with amazing breakthroughs and techniques, but both come from the more traditional and formal style of academic paper writing, which relies on lots of heavy mathematical proofs. I don't have a mathematical or technical background, so unfortunately I don't know how to put their rigorous techniques into practice in a GLSL shader. But it is worth looking at both of these important works in order to get a general, bird's-eye view of why and how they proposed this type of path tracing in the first place.

As far as more modern resources go, I have found a couple that simplify the language a little, and even a ShaderToy that 'correctly'(?) implements basic BDPT, with appropriate weight calculations (as far as I can tell). Here is a short list that may be of help:

https://www.sjbrown.co.uk/posts/two-way-path-tracing/ https://graphics.pixar.com/library/BiDir/paper.pdf https://agraphicsguy.wordpress.com/2016/01/16/practical-implementation-of-mis-in-bidirectional-path-tracing/ https://www.shadertoy.com/view/4lfGWr

A note on my own version of BDPT: as mentioned, I don't really do the full BDPT method, with all its path weight calculations (pdfs) and such. However, I do weight both paths according to the Lambert cosine model, which is a very simple dot product term between the eye/light ray paths and the surface normals that they happen to strike. Although I am attenuating the energy of both paths, this is probably a gross underestimate of the true Monte Carlo pdf weights needed for connecting such eye and light paths at their vertices. Therefore, I wouldn't want to put my final images up against something done in a professional render like Lux or RenderMan, because I can guarantee there would be discrepancies (more like errors on my part) between the two.

Instead, I went for a more custom, empirical approach. And if you'll allow me, I'll do the best I can to walk you through the steps of how I handle basic BDPT.

From your original post, you are spot-on with your understanding and assumptions of what's going on. Basically, every animation frame we send out a single random ray from the light source and follow it for a random lifetime (could be 1 bounce, could be 2 or 3, however many you're willing to wait for in a real-time shader). I also found out the hard way, that sometimes the lifetime needs to be 0, or no rays - that way, once in a while the eye path can sample the lights just how we always do in a regular path tracer (otherwise, surfaces around the light that can directly see it are underlit and under-represented).

At first I tried generating uniformly random light rays emanating from the light sources, but along the way, I realized a cool trick that speeds up convergence: for spot lights and directed lights, send them out just like you would a diffuse surface sampler - good old Lambertian cosine-weighted ray directions! Don't know if anyone else does that, but it seems to help. :-)

Here is where my version differs most: if the random light path ray hits a specular surface (like a mirror or transparent glass), I keep bouncing. As soon as the light path ray hits its first (or second, by random choice) diffuse surface, I stop the bounces loop and record the hitPoint. I move this point out a little along the diffuse surface's normal (to avoid shadow acne due to floating point precision inaccuracies), and later I will treat it as a light source to sample, just like any other path tracer would sample a point light source. I also attenuate the light path's energy whenever it strikes a diffuse surface, using the familiar cosine (N . L) diffuse shading model.

That concludes the 1st bounces loop (the light bounces loop). Now comes the regular eye (or camera) bounces loop, which is almost identical to a normal, unidirectional path tracer. The difference lies in handling all diffuse (or rough specular) surfaces. Instead of sampling the light source directly, which would result in many failures if the light sources are hidden in any way or require complex paths to get their light out into the scene, we sample the earlier recorded light hitPoint (that is only attached to various diffuse surfaces, recall). This essentially connects the eye and the light paths together. Note that I randomly choose only 1 terminal vertex on each path to connect both paths with - in the original BDPT papers, it is advised to connect ALL vertices from both paths (could be dozens of connections, like traveling salesman problem, ha ha), and then cleverly use MIS to choose and properly weight the best one with the most contributions along the newly connected path.

One more detail about connecting the eye and the light paths: for each connection attempt, you must send out a shadow ray (or direct point light sample) towards the earlier recorded light hitPoint. If using the full BDPT method, all of these interconnecting shadow rays would really add up and would bog down a real-time shader. So I only do 1 shadow ray for 1 random connection attempt between the two paths. The good news is that I can still achieve 30 to 60 fps and get the hard-to-reach light to come out into the scene a little more - the bad news is: heavy noise (because of my naïve algorithm), and if that one precious connection isn't able to be made (an object or surface is in between the eye path point and the light path point), it necessarily returns a black pixel and all of our setup and hard work for this pixel was for nothing. :(

Having said that, most of the random single connections are able to be made, and I quite like the empirical results, even if they may not be mathematically/statistically sound. Feel free to check out my GLSL shader for my Bi-Directional Path Tracing demo. I tried to name the variables carefully so that they reflect what their purpose is, and I think I added a couple of comments here and there.

Hope you can get something going with BDPT in your own project here. Sorry I can't be more rigorous in my approach/explanation. But I'll be glad to answer any questions you might have and will try to help as best as I can. :-)

-Erich

[gkjohnson]

Excellent - thank you for the overview! I'll take a look at some of the papers before getting into it. It's probably a bit of a ways out before I'll be putting time into this but this is a good head start!