Orthograpic camera results in empty normal output

[chenzhekl]

I am using the combination of an orthographic camera and a framebuffer with OSP_FB_COLOR | OSP_FB_NORMAL flags. The output normal in this setting is always all zeros.

[johguenther]

Hmm, I cannot reproduce this, normals are written as expected also with the orthographic camera. I modified ospExamples to use orthographic camera and show normals when "show albedo" is clicked:

diff --git a/apps/ospExamples/GLFWOSPRayWindow.cpp b/apps/ospExamples/GLFWOSPRayWindow.cpp
index 3b7f5271b..533fa41ce 100644
--- a/apps/ospExamples/GLFWOSPRayWindow.cpp
+++ b/apps/ospExamples/GLFWOSPRayWindow.cpp
@@ -280,6 +280,7 @@ void GLFWOSPRayWindow::reshape(const vec2i &newWindowSize)
 void GLFWOSPRayWindow::updateCamera()
 {
   camera.setParam("aspect", windowSize.x / float(windowSize.y));
+  camera.setParam("height", 30.0f);
   camera.setParam("position", arcballCamera->eyePos());
   camera.setParam("direction", arcballCamera->lookDir());
   camera.setParam("up", arcballCamera->upDir());
@@ -343,7 +344,7 @@ void GLFWOSPRayWindow::display()
     latestFPS = 1.f / currentFrame.duration();

     auto *fb = framebuffer.map(
-        showDepth ? OSP_FB_DEPTH : (showAlbedo ? OSP_FB_ALBEDO : OSP_FB_COLOR));
+        showDepth ? OSP_FB_DEPTH : (showAlbedo ? OSP_FB_NORMAL : OSP_FB_COLOR));

     glBindTexture(GL_TEXTURE_2D, framebufferTexture);
     glTexImage2D(GL_TEXTURE_2D,
diff --git a/apps/ospExamples/GLFWOSPRayWindow.h b/apps/ospExamples/GLFWOSPRayWindow.h
index 493f87b6f..e0b3d4a4a 100644
--- a/apps/ospExamples/GLFWOSPRayWindow.h
+++ b/apps/ospExamples/GLFWOSPRayWindow.h
@@ -77,7 +77,7 @@ class GLFWOSPRayWindow
   cpp::Renderer rendererAO{"ao"};
   cpp::Renderer rendererDBG{"debug"};
   cpp::Renderer *renderer{nullptr};
-  cpp::Camera camera{"perspective"};
+  cpp::Camera camera{"orthographic"};
   cpp::World world;
   cpp::Light sunSky{"sunSky"};
   cpp::FrameBuffer framebuffer;

Works with pathtracer and scivis renderer (although I noted a bug in scivis, which outputs normals for the background).

[chenzhekl]

Hi @johguenther, Thanks for your prompt reply! I am sorry that I am fully occupied with my work and cannot dig into this problem. Here is the snippet I used for reproducing the problem. I hope it can help you.

code.tar.gz

[johguenther]

Ok, here is what's going on: normal and albedo buffers are primarily for denoising. So the pathtracer writes those not literally on "the first hit", but on "the first non-specular hit", to focus denoising of objects seen behind windows or in mirrors. Now, in your reproducer you set the material to be fully metallic, i.e. everything is a mirror, thus normals don't ever get written. Using a material with diffuse components (e.g. metallic = 0.1) results in normals outputted.

We need to clarify this behavior in the documentation.

[chenzhekl]

Thank you @johguenther ! That clarifies things a lot. But I still feel that recordung albedo, normal, etc for the first hit is a better idea. Please correct me if I am wrong. For one, even though the reflection of a specular object alone is niose-free, the noise from other diffuse objects can still be baked into the mirror object through indirect illumination. For the other, in computer vision, we often need to generate things like first-hit normal as part of the synthetic dataset. Being able to produce such data like Mitsuba could be a big plus.

Btw, in case if you are interested, changing the orthographic camera to be perspective in the reproduction code makes ospray outout the normal, which is inconsistent.

[johguenther]

We think about adding a "first hit" normal buffer.

For one, even though the reflection of a specular object alone is niose-free, the noise from other diffuse objects can still be baked into the mirror object through indirect illumination.

This is exactly the reason why the albedo (and normal) buffer for denoising use the information potentially from secondary hits (the reflected objects), to have better denoising (less blur) of objects seen behind glass or via a mirror. To clarify: You are primarily interested in the "first hit" normals; no need for "first hit" albedo (in addition to the current denoising albedo)?

[johguenther]

Commit ff3b37428fa6c67fd8e6d7039112d8158764c2fd adds another framebuffer channel OSP_FB_FIRST_NORMAL, which always holds the first hit world-space normal (regardless of material reflection or transparency).