mitsuba-renderer / mitsuba3

Mitsuba 3: A Retargetable Forward and Inverse Renderer
https://www.mitsuba-renderer.org/
Other
2.1k stars 246 forks source link

Calculation of discontinuous derivatives of visibility #1184

Closed ejtwe closed 5 months ago

ejtwe commented 6 months ago

Summary

There is a difference between the render_primarily_visible_silhouette function in the common.py file and the derivative calculation in the paper.

Description

In the render_primarily_visible_silhouette function of common.py:

      ““”
      def render_primarily_visible_silhouette(self,
                                                  scene: mi.Scene,
                                                  sensor: mi.Sensor,
                                                  sampler: mi.Sampler,
                                                  spp: int) -> mi.TensorXf:
              """
              Renders the primarily visible discontinuities.

              This method returns the AD-attached image. The result must still be
              traversed using one of the Dr.Jit functions to propagate gradients.
              """
              film = sensor.film()
              aovs = self.aov_names()

              # Explicit sampling to handle the primarily visible discontinuous derivative
              with dr.suspend_grad():
                  # Get the viewpoint
                  sensor_center = sensor.world_transform() @ mi.Point3f(0)

                  # Sample silhouette point
                  ss = self.proj_detail.sample_primarily_visible_silhouette(
                      scene, sensor_center, sampler.next_2d(), True)
                  active = ss.is_valid() & (ss.pdf > 0)

                  # Jacobian (motion correction included)
                  J = self.proj_detail.perspective_sensor_jacobian(sensor, ss)

                  ΔL = self.proj_detail.eval_primary_silhouette_radiance_difference(
                      scene, sampler, ss, sensor_center, active=active)
                  active &= dr.any(dr.neq(ΔL, 0))

              # ∂z/∂ⲡ * normal
              si = dr.zeros(mi.SurfaceInteraction3f)
              si.p = ss.p
              si.prim_index = ss.prim_index
              si.uv = ss.uv
              p = ss.shape.differential_motion(dr.detach(si), active)
              motion = dr.dot(p, ss.n)

              # Compute the derivative
              derivative = ΔL * motion * dr.rcp(ss.pdf) * J
             ......
           ”“”

The relevant formulas in the paper are as follows: 微信截图_20240526224921

---But I don't see the sinφ item. ---And p in "p = ss.shape.differential_motion(dr.detach(si), active); motion = dr.dot(p, ss.n)" is the barycenter coordinate? Does it correspond to 𝜕𝜃 xb in the paper? ---Can the calculation of the Jacobian J provide a more detailed derivation or explanation? (function perspective_sensor_jacobian())

Thank you very much for your answer!