Error camera_up value c(0,1,0) is aligned exactly with camera vector when rendering orthogonal view

[wiesehahn]

I try to render a scene with rayshader::render_highquality() and receive the following error after the process is running for some time.

--------------------------Interactive Mode Controls---------------------------
W/A/S/D: Horizontal Movement: | Q/Z: Vertical Movement | Up/Down: Adjust FOV | ESC: Close
Left/Right: Adjust Aperture  | 1/2: Adjust Focal Distance | 3/4: Rotate Environment Light 
P: Print Camera Info | R: Reset Camera |  TAB: Toggle Orbit Mode |  E/C: Adjust Step Size
K: Save Keyframe | L: Reset Camera to Last Keyframe (if set) | F: Toggle Fast Travel Mode
Left Mouse Click: Change Look At (new focal distance) | Right Mouse Click: Change Look At 
Error in prepare_scene_list(scene = scene, width = width, height = height,  : 
  camera_up value c(0,1,0) is aligned exactly with camera vector (lookat - lookfrom). Choose a different value for camera_up.

I guess this related to the fact that I try to render an orthogonal view, since a slight change to phi=89 works.

The code looks like

elmat %>%
  sphere_shade(zscale = 0.25) %>%
  add_shadow(ray_shade(elmat), 0.6) %>%
  plot_3d(elmat, zscale = 0.25, fov = 45, theta = 0, zoom = 0.8, phi = 90, windowsize = c(500, 500), solid=F) 

render_highquality(
                   samples = 300, 
                   parallel = TRUE,
                   light = FALSE,
                   rotate_env =  180,
                   clear = TRUE,
                   width = 1000,
                   height = 1000,
                   filename = here("images", "filename.png")
)

Setting e.g. camera_up = c(0,1.1,0) in render_highquality() does not resolve the error.

[tylermorganwall]

Hi @wiesehahn, thank you for bringing up this issue and for the detailed information you provided.

The error you're encountering is expected behavior in this case. When the camera_up vector is exactly aligned with the lookfrom-lookat vector, the coordinate system becomes ill-defined. {rgl} does not provide an easy way to extract the complete orientation of the camera when phi = 90, so rayshader can not tell rayrender what the current value of theta is and thus cannot provide an appropriate vector for camera_up.

The root of the issue lies in how the camera_up and lookfrom-lookat vectors contribute to defining an orthonormal basis for the camera's coordinate system in the rendered scene. These vectors are usually orthogonalized to produce three mutually perpendicular unit vectors that define the "up," "right," and "forward" directions of the camera. When the camera_up and lookfrom-lookat vectors are perfectly aligned, they essentially become linearly dependent, making it impossible to form a complete orthonormal basis. This results in a mathematical ambiguity in the camera's coordinate system, which prevents the renderer from effectively determining the "up" and "right" directions for the camera.

There are two ways to resolve this issue:

1. Specify an Appropriate camera_up Vector: Consider setting it to a value that makes sense for your visualization. For instance, if the z-axis is the "up" direction in your map, set camera_up to c(0, 0, 1).

   render_highquality(
     camera_up = c(0, 0, 1),
     samples = 300, 
     parallel = TRUE,
     light = FALSE,
     rotate_env = 180,
     clear = TRUE,
     width = 1000,
     height = 1000,
     filename = here("images", "filename.png")
   )

2. Adjust the Camera Angle (phi): You mentioned that changing the camera angle to phi = 89 worked for you. You can make a smaller adjustment to phi = 89.9 if you want the view to be almost orthogonal, yet avoid the issue.

   plot_3d(elmat, zscale = 0.25, fov = 45, theta = 0, zoom = 0.8, phi = 89.9, windowsize = c(500, 500), solid = F)

[tylermorganwall]

I believe this issue is resolved on the rayrender side--if there is a change of behavior or you think there's a better way to handle these angles, feel free to open an issue on the rayshader package.