search

NVIDIAGameWorks / RTX-Path-Tracing

Real-time path tracing library and sample
Other
599 stars 64 forks source link

Path-Tracing-SDK for VR? #18

Closed ghmechamania closed 5 months ago

ghmechamania commented 10 months ago

Hi,

We tried to use RTXDI with NRD to build a full ray traced VR engine. But because of the boiling effect of NRD we did abandon the project. The boiling effect is more or less random and thus different for left and right eye. This killed the immersion and we could not solve that. But now we have seen a demo of DLSS 3.5 and the boiling effect seems completely vanished! See this video at 0:42.

So my first question is: When will DLSS 3.5 be integrated in this Path-Tracing SDK?

And because this makes path traced VR possible, we have the following questions:

With DLSS 3.5, support for OpenXR and foveated rendering, good quality path traced should finally be possible!

tiont commented 5 months ago

Hi @ghmechamania. The latest version of the path tracing SDK includes support for DLSS 3.5. You can enable DLSS-FG (frame generation) in the GUI. Currently, we do not have a timeline for OpenXR or foveated rendering support. Please stay tuned for any updates on those features in future SDK releases. Thank you!

fstrugar-nv commented 5 months ago

Just a note - we still rely on NRD for denoising - the DLSS-RR is not yet available for general use, thus the boiling problem still remains in PTSDK. However, we do have lighting improvements which reduces boiling/image instability. And we hope to release -RR in PTSDK soon.

One note: we can't guarantee that DLSS-RR will not have similar (although a lot smaller) discrepancy between views - that's something to be tested yet.

Also I think there should be a way to "sync up" the left/right eye views when using NRD in such way that you would find a matching texel from the other view based on worldspace position and, if match available, lerp towards it by some percentage (say 20%). It's just an use case we haven't yet tested unfortunately.