implementation of p5.Camera.slerp(), and minor specification change of orbitControl()

[inaridarkfox4231]

Increasing Access

If there are two cameras, I'd like to implement a function to interpolate each camera's view. For example, if one camera is facing the front and the other is facing the right, the interpolated camera will be facing diagonally to the right. It's easier to understand with an example, so I'll give you a few examples. camera.slerp()_sample1

let cam0, cam1, cam;
function setup() {
  createCanvas(100, 100, WEBGL);
  strokeWeight(3);

  // camera for slerp.
  cam = createCamera();
  // cam0 is looking at the cube from the front.
  // On the other hand, cam1 is pointing straight to the right
  // at the same position as cam0.
  cam0 = createCamera();
  cam0.camera(0, 0, 90, 0, 0, 0, 0, 1, 0);
  cam1 = createCamera();
  cam1.camera(0, 0, 90, 90, 0, 90, 0, 1, 0);

  // we only use cam.
  setCamera(cam);
}

function draw() {
  // calculate amount.
  const amt = 0.5 - 0.5 * cos(frameCount * TAU / 120);
  // slerp cam0 and cam1 with amt, set to cam.
  // When amt moves from 0 to 1, cam moves from cam0 to cam1,
  // shaking the camera to the right.
  cam.slerp(cam0, cam1, amt);

  background(255);
  // Every time the camera turns right, the cube drifts left.
  box(40);
}

https://github.com/processing/p5.js/assets/39549290/5af19d8c-9251-40c0-84b1-621364100a66

cam0 is looking straight ahead at the cube from above. On the other hand, cam1 is at the same position as cam0 and facing straight to the right of the cube. Interpolating these between 0 and 1 and setting them to cam will smoothly interpolate the view between them. Only cam is active. slerp() will update uMV and uP if the target is active, so there is no need to call setCamera(). I had set() implemented to achieve this.

Another example concerns orbitControl(). camera.slerp()_sample2

let cam, lastCam, initialCam;
let countForReset = 30;
// This sample uses orbitControl() to move the camera.
// Double-clicking the canvas restores the camera to its initial state.
function setup() {
  createCanvas(100, 100, WEBGL);
  strokeWeight(3);

  cam = createCamera(); // main camera
  lastCam = createCamera(); // Camera for recording loc info before reset
  initialCam = createCamera(); // Camera for recording the initial state

  setCamera(cam); // set main camera
}

function draw() {
  if (countForReset < 30) {
    // if the reset count is less than 30,
    // it will move closer to the original camera as it increases.
    countForReset++;
    cam.slerp(lastCam, initialCam, countForReset / 30);
  } else {
    // if the count is 30,
    // you can freely move the main camera with orbitControl().
    orbitControl();
  }

  background(255);
  box(40);
}
// A double-click sets countForReset to 0 and initiates a reset.
function doubleClicked() {
  if (countForReset === 30) {
    countForReset = 0;
    lastCam.set(cam);
  }
}

https://github.com/processing/p5.js/assets/39549290/f6ceb617-0649-48d2-aa40-ca7d4922f5a8

Prepare two cameras, initialCam and lastCam. Set the initial state in initialCam. lastCam is used to record the current state of cam when resetting the camera. Double-clicking resets the counter to 0, and until it reaches a constant value, interpolates the view from the initial state and sets it to the camera. Also in this case, no setCamera() is needed as only cam is active. The nice thing about slerp() is that you can chain multiple views without explicitly switching cameras.

Also, some fixes for orbitControl().

    // In orthogonal projection, the scale does not change even if
    // the distance to the gaze point is changed, so the projection matrix
    // needs to be modified.
    if (cam.projMatrix.mat4[15] !== 0) {
      cam.projMatrix.mat4[0] *= Math.pow(
        10, -this._renderer.zoomVelocity
      );
      cam.projMatrix.mat4[5] *= Math.pow(
        10, -this._renderer.zoomVelocity
      );
      // modify uPMatrix
      this._renderer.uPMatrix.mat4[0] = cam.projMatrix.mat4[0];
      this._renderer.uPMatrix.mat4[5] = cam.projMatrix.mat4[5];
    }

We rewrite uP internally so that it can be scaled with ortho(), but this not change the camera's projMatrix. Therefore, first rewrite projMatrix and rewrite it so that its contents are reflected in uP.

  const mouseZoomScaleFactor = 0.01;
/* ~~~~~~~~ */
      // zoom the camera depending on the value of _mouseWheelDeltaY.
      // move away if positive, move closer if negative
      deltaRadius = Math.sign(this._mouseWheelDeltaY) * sensitivityZ;
      deltaRadius *= mouseZoomScaleFactor;

If the value of mouseWheelDeltaY is used as it is, the behavior may change due to the difference in value depending on the model. So, I decided to use a constant which multiplied the sign of mouseWheelDeltaY. Behavior remained almost unchanged. I think there is no problem.

Also, I fixed several places where I could write in the global variable cam.

Most appropriate sub-area of p5.js?

• [ ] Accessibility
• [ ] Color
• [ ] Core/Environment/Rendering
• [ ] Data
• [ ] DOM
• [ ] Events
• [ ] Image
• [ ] IO
• [ ] Math
• [ ] Typography
• [ ] Utilities
• [X] WebGL
• [ ] Build Process
• [ ] Unit Testing
• [ ] Internalization
• [ ] Friendly Errors
• [ ] Other (specify if possible)

Feature request details

The implementation of slerp() is as follows:

  /**
 * For the cameras cam0 and cam1 with the given arguments, their view are combined
 * with the parameter amt that represents the quantity, and the obtained view is applied.
 * For example, if cam0 is looking straight ahead and cam1 is looking straight
 * to the right and amt is 0.5, the applied camera will look to the halfway
 * between front and right.
 * If the applied camera is active, the applied result will be reflected on the screen.
 * When applying this function, the projection modes of all appearing cameras
 * must match. For example, if one is perspective, ortho, frustum, the other
 * two must also be perspective, ortho, frustum respectively.
 *
 * @method slerp
 * @param {p5.Camera} cam0 first p5.Camera
 * @param {p5.Camera} cam1 second p5.Camera
 * @param {Number} amt amount to use for interpolation during slerp
 *
 * @example
 * <div>
 * <code>
 * let cam0, cam1, cam;
 * function setup() {
 *   createCanvas(100, 100, WEBGL);
 *   strokeWeight(3);
 *
 *   // camera for slerp.
 *   cam = createCamera();
 *   // cam0 is looking at the cube from the front.
 *   // On the other hand, cam1 is pointing straight to the right
 *   // at the same position as cam0.
 *   cam0 = createCamera();
 *   cam0.camera(0, 0, 90, 0, 0, 0, 0, 1, 0);
 *   cam1 = createCamera();
 *   cam1.camera(0, 0, 90, 90, 0, 90, 0, 1, 0);
 *
 *   // we only use cam.
 *   setCamera(cam);
 * }
 *
 * function draw() {
 *   // calculate amount.
 *   const amt = 0.5 - 0.5 * cos(frameCount * TAU / 120);
 *   // slerp cam0 and cam1 with amt, set to cam.
 *   // When amt moves from 0 to 1, cam moves from cam0 to cam1,
 *   // shaking the camera to the right.
 *   cam.slerp(cam0, cam1, amt);
 *
 *   background(255);
 *   // Every time the camera turns right, the cube drifts left.
 *   box(40);
 * }
 * </code>
 * </div>
 * @alt
 * Prepare two cameras. One camera is facing straight ahead to the cube and the other
 * camera is in the same position and looking straight to the right.
 * If you use a camera which interpolates these with slerp(), the facing direction
 * of the camera will change smoothly between the front and the right.
 *
 * @example
 * <div>
 * <code>
 * let cam, lastCam, initialCam;
 * let countForReset = 30;
 * // This sample uses orbitControl() to move the camera.
 * // Double-clicking the canvas restores the camera to its initial state.
 * function setup() {
 *   createCanvas(100, 100, WEBGL);
 *   strokeWeight(3);
 *
 *   cam = createCamera(); // main camera
 *   lastCam = createCamera(); // Camera for recording loc info before reset
 *   initialCam = createCamera(); // Camera for recording the initial state
 *
 *   setCamera(cam); // set main camera
 * }
 *
 * function draw() {
 *   if (countForReset < 30) {
 *     // if the reset count is less than 30,
 *     // it will move closer to the original camera as it increases.
 *     countForReset++;
 *     cam.slerp(lastCam, initialCam, countForReset / 30);
 *   } else {
 *     // if the count is 30,
 *     // you can freely move the main camera with orbitControl().
 *     orbitControl();
 *   }
 *
 *   background(255);
 *   box(40);
 * }
 * // A double-click sets countForReset to 0 and initiates a reset.
 * function doubleClicked() {
 *   if (countForReset === 30) {
 *     countForReset = 0;
 *     lastCam.set(cam);
 *   }
 * }
 * </code>
 * </div>
 * @alt
 * There is a camera, drawing a cube. The camera can be moved freely with
 * orbitControl(). Double-click to smoothly return the camera to its initial state.
 * The camera cannot be moved during that time.
 */
  slerp(cam0, cam1, amt) {
    // If t is 0 or 1, do not interpolate and set the argument camera.
    if (amt === 0) {
      this.set(cam0);
      return;
    } else if (amt === 1) {
      this.set(cam1);
      return;
    }

    // For this cameras is ortho, assume that cam0 and cam1 are also ortho
    // and interpolate the elements of the projection matrix.
    if (this.projMatrix.mat4[15] !== 0) {
      this.projMatrix.mat4[0] =
        (1 - amt) * cam0.projMatrix.mat4[0] + amt * cam1.projMatrix.mat4[0];
      this.projMatrix.mat4[5] =
        (1 - amt) * cam0.projMatrix.mat4[5] + amt * cam1.projMatrix.mat4[5];
    }
    // If the camera is active, make uPMatrix reflect changes in projMatrix.
    if (this._isActive()) {
      this._renderer.uPMatrix.mat4 = this.projMatrix.mat4.slice();
    }

    // Linearly interpolates the distance between the viewpoint and the center,
    // and the position of the center.
    const eyeDist0 = Math.hypot(
      cam0.eyeX - cam0.centerX, cam0.eyeY - cam0.centerY, cam0.eyeZ - cam0.centerZ
    );
    const eyeDist1 = Math.hypot(
      cam1.eyeX - cam1.centerX, cam1.eyeY - cam1.centerY, cam1.eyeZ - cam1.centerZ
    );
    const eyeDist = (1 - amt) * eyeDist0 + amt * eyeDist1;
    const lerpedCenterX = (1 - amt) * cam0.centerX + amt * cam1.centerX;
    const lerpedCenterY = (1 - amt) * cam0.centerY + amt * cam1.centerY;
    const lerpedCenterZ = (1 - amt) * cam0.centerZ + amt * cam1.centerZ;

    // Prepare each of rotation matrix from their camera matrix
    const mat0 = cam0.cameraMatrix.mat4;
    const mat1 = cam1.cameraMatrix.mat4;
    const m0 = [
      mat0[0], mat0[1], mat0[2],
      mat0[4], mat0[5], mat0[6],
      mat0[8], mat0[9], mat0[10]
    ];
    const m1 = [
      mat1[0], mat1[1], mat1[2],
      mat1[4], mat1[5], mat1[6],
      mat1[8], mat1[9], mat1[10]
    ];
    // Create the inverse matrix of mat0 by transposing mat0,
    // and multiply it to mat1 from the right.
    // This matrix represents the difference between the two.
    const m = [
      m1[0] * m0[0] + m1[1] * m0[1] + m1[2] * m0[2],
      m1[0] * m0[3] + m1[1] * m0[4] + m1[2] * m0[5],
      m1[0] * m0[6] + m1[1] * m0[7] + m1[2] * m0[8],

      m1[3] * m0[0] + m1[4] * m0[1] + m1[5] * m0[2],
      m1[3] * m0[3] + m1[4] * m0[4] + m1[5] * m0[5],
      m1[3] * m0[6] + m1[4] * m0[7] + m1[5] * m0[8],

      m1[6] * m0[0] + m1[7] * m0[1] + m1[8] * m0[2],
      m1[6] * m0[3] + m1[7] * m0[4] + m1[8] * m0[5],
      m1[6] * m0[6] + m1[7] * m0[7] + m1[8] * m0[8]
    ];

    // Calculate the trace and from it the cos value of the angle.
    // trace = 1 + 2 * cosTheta.
    let cosTheta = 0.5 * (m[0] + m[4] + m[8] - 1);
    // If the angle is close to 0, the two matrices are very close,
    // so in that case we execute linearly interpolate.
    if (1 - cosTheta < 0.0000001) {
      // Obtain the front vector and up vector by linear interpolation
      // and normalize them.
      const lerpedFront = new p5.Vector(
        (1 - amt) * m0[2] + amt * m1[2],
        (1 - amt) * m0[5] + amt * m1[5],
        (1 - amt) * m0[8] + amt * m1[8]
      ).normalize();
      const lerpedUp = new p5.Vector(
        (1 - amt) * m0[1] + amt * m1[1],
        (1 - amt) * m0[4] + amt * m1[4],
        (1 - amt) * m0[7] + amt * m1[7]
      ).normalize();
      // set the camera
      this.camera(
        lerpedCenterX + lerpedFront.x * eyeDist,
        lerpedCenterY + lerpedFront.y * eyeDist,
        lerpedCenterZ + lerpedFront.z * eyeDist,
        lerpedCenterX, lerpedCenterY, lerpedCenterZ,
        lerpedUp.x, lerpedUp.y, lerpedUp.z
      );
      return;
    }

    // Calculates the axis vector and the angle of the difference orthogonal matrix.
    // similar calculation is here:
    // https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294
    let a, b, c, sinTheta;
    let invOneMinusCosTheta = 1 / (1 - cosTheta);
    if (m[0] > m[4] && m[0] > m[8]) {
      a = Math.sqrt((m[0] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= a;
      b = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
      c = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[7] - m[5]) / a;
    } else if (m[4] > m[8]) {
      b = Math.sqrt((m[4] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= b;
      c = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
      a = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[2] - m[6]) / b;
    } else {
      c = Math.sqrt((m[8] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= c;
      a = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
      b = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[3] - m[1]) / c;
    }

    // Constructs a new matrix after interpolating the angles.
    // Multiplying mat0 by the first matrix yields mat1, but by creating a state
    // in the middle of that matrix, you can obtain a matrix that is
    // an intermediate state between mat0 and mat1.
    const angle = amt * Math.atan2(sinTheta, cosTheta);
    const cosAngle = Math.cos(angle);
    const sinAngle = Math.sin(angle);
    const oneMinusCosAngle = 1 - cosAngle;
    const ab = a * b;
    const bc = b * c;
    const ca = c * a;
    const result = [
      cosAngle + oneMinusCosAngle * a * a,
      oneMinusCosAngle * ab - sinAngle * c,
      oneMinusCosAngle * ca + sinAngle * b,
      oneMinusCosAngle * ab + sinAngle * c,
      cosAngle + oneMinusCosAngle * b * b,
      oneMinusCosAngle * bc - sinAngle * a,
      oneMinusCosAngle * ca - sinAngle * b,
      oneMinusCosAngle * bc + sinAngle * a,
      cosAngle + oneMinusCosAngle * c * c
    ];

    // Multiply this to mat0 from left to get the interpolated front vector.
    const front = new p5.Vector(
      result[0] * m0[2] + result[1] * m0[5] + result[2] * m0[8],
      result[3] * m0[2] + result[4] * m0[5] + result[5] * m0[8],
      result[6] * m0[2] + result[7] * m0[5] + result[8] * m0[8]
    );

    // We also get the up vector in the same way and set the camera.
    this.camera(
      lerpedCenterX + front.x * eyeDist,
      lerpedCenterY + front.y * eyeDist,
      lerpedCenterZ + front.z * eyeDist,
      lerpedCenterX, lerpedCenterY, lerpedCenterZ,
      result[0] * m0[1] + result[1] * m0[4] + result[2] * m0[7],
      result[3] * m0[1] + result[4] * m0[4] + result[5] * m0[7],
      result[6] * m0[1] + result[7] * m0[4] + result[8] * m0[7]
    );
  }

It takes two cameras and interpolates their views. If the target camera is active, it will be reflected in uMV and uP and the screen will be updated. All camera projection modes must match. For example, you cannot interpolate a perspective camera and an ortho camera. The projection matrix must also match, but only in the case of ortho(), interpolation is possible if only the 0th or 5th component differs due to orbitControl(). If amt is 0 or 1, the argument camera is set as is.

It looks like there are a lot of redundant calculations involving 3x3 matrices, but the original code had a lot more, so I considered making them into methods. However, in the process of simplification and speedup of the method, it was reduced and almost disappeared. Therefore, it was decided that it would be better to write directly rather than making it a method, but I don't know if this is okay.

In addition to these, I plan to implement the orbitControl() fix I mentioned above.

[inaridarkfox4231]

The first example may be easier to understand using pan(). I think I will rewrite it.

[inaridarkfox4231]

I decided to rewrite setup() in the first example as follows:

function setup() {
  createCanvas(100, 100, WEBGL);
  strokeWeight(3);

  // camera for slerp.
  cam = createCamera();
  // cam0 is looking at the cube from the front.
  // cam1 is pointing straight to the right in the cube
  // at the same position as cam0 by doing a pan(-PI/2).
  cam0 = createCamera();
  cam1 = createCamera();
  cam1.pan(-PI/2);

  // we only use cam.
  setCamera(cam);
}

I thought this was easier to understand than specifying it with the camera() function.

I tried to execute it with tilt(), but I gave up because it broke the camera when tilt(±PI/2) is executed because of the specification that does not rewrite the up vector. I can't decide at this time if this should be fixed, so I'm putting it on hold. Even if it should be addressed, it is not covered in this issue. I think that even if the up vector is also rotated at the same time, the problem will not occur, but I don't know. If I deal with it, I think I'll create a new issue.

Specifically, if you do this in the example of tilt(), the camera will not return.

https://github.com/processing/p5.js/assets/39549290/4ee7a2dd-8230-4f39-9498-72a959611f08

[inaridarkfox4231]

If tilt() is specified so that the up vector changes, the behavior of pan() after tilt() will change. I don't know if this is good or bad, so I'm not going to touch on this issue.

I will add the results of the benchmark test instead.

benchmark test: 12th Gen Intel(R) Core(TM) i7-1260P 2.10 GHz Windows 11 Home

After 60 frames with 50000 calls per frame and taking the average, the processing speed did not drop at all. Actually, I think it's a few times at most, but I was able to confirm that it was fast enough.

[davepagurek]

I think this is a useful function to add!

I wonder if we can leverage the slerp code you added to p5.Vector to reduce the code complexity in p5.Camera and make it easier for future contributors to see what's going on? e.g. doing something like this:

const eye0 = createVector(cam0.eyeX, cam0.eyeY, cam0.eyeZ);
const eye1 = createVector(cam1.eyeX, cam1.eyeY, cam1.eyeZ);
const center0 = createVector(cam0.centerX, cam0.centerY, cam0.centerZ);
const center1 = createVector(cam1.centerX, cam1.centerY, cam1.centerZ);
const up0 = createVector(cam0.upX, cam0.upY, cam0.upZ);
const up1 = createVector(cam1.upX, cam1.upY, cam1.upZ);
const newCenter = center0.copy().lerp(center1, amt);
const newLook = center0.copy().sub(eye0).slerp(center1.copy().sub(eye1), amt);
const newEye = newCenter.copy().sub(newLook);
const newUp = up0.copy().slerp(up1, amt);
this.camera(
  newEye.x, newEye.y, newEye.z,
  newCenter.x, newCenter.y, newCenter.z,
  newUp.x, newUp.y, newUp.z
);

Quick copy of your sketch with this code pasted in: https://editor.p5js.org/davepagurek/sketches/wkLdQ9DUT

I haven't thought through all the implications of this, so if there are cases that the matrix math handles that this doesn't, there's nothing wrong with using the version you suggested.

[inaridarkfox4231]

Unfortunately, I cannot accept that proposal. See what happens when you rotate 180 degrees vertically.

https://github.com/processing/p5.js/assets/39549290/ec977c09-3c4b-42f8-9625-53edf3189fb2

I wanted to introduce vector slerp() either purely out of curiosity or convenience. Not for cameras. At first, I was confused between vector slerp and camera slerp, or more precisely, quaternion slerp. But the two are actually very different. Even if only the vector is interpolated with slerp(), the intermediate process will not become an orthonormal basis, so the interpolation will be messy. So it doesn't work. This proposal is useless, sorry.

I think the next part is probably the hardest to understand.

    // Calculates the axis vector and the angle of the difference orthogonal matrix.
    // similar calculation is here:
    // https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294
    let a, b, c, sinTheta;
    let invOneMinusCosTheta = 1 / (1 - cosTheta);
    if (m[0] > m[4] && m[0] > m[8]) {
      a = Math.sqrt((m[0] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= a;
      b = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
      c = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[7] - m[5]) / a;
    } else if (m[4] > m[8]) {
      b = Math.sqrt((m[4] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= b;
      c = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
      a = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[2] - m[6]) / b;
    } else {
      c = Math.sqrt((m[8] - cosTheta) * invOneMinusCosTheta); // not zero.
      invOneMinusCosTheta /= c;
      a = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
      b = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
      sinTheta = 0.5 * (m[3] - m[1]) / c;
    }

What we're doing here is that we're assuming some form of orthogonal matrix, and we're finding the variables that appear in it by comparing the coefficients. You'll see what it looks like in the next part.

    const angle = amt * Math.atan2(sinTheta, cosTheta);
    const cosAngle = Math.cos(angle);
    const sinAngle = Math.sin(angle);
    const oneMinusCosAngle = 1 - cosAngle;
    const ab = a * b;
    const bc = b * c;
    const ca = c * a;
    const result = [
      cosAngle + oneMinusCosAngle * a * a,
      oneMinusCosAngle * ab - sinAngle * c,
      oneMinusCosAngle * ca + sinAngle * b,
      oneMinusCosAngle * ab + sinAngle * c,
      cosAngle + oneMinusCosAngle * b * b,
      oneMinusCosAngle * bc - sinAngle * a,
      oneMinusCosAngle * ca - sinAngle * b,
      oneMinusCosAngle * bc + sinAngle * a,
      cosAngle + oneMinusCosAngle * c * c
    ];

I'm looking for a comparison with this. This is due to the nature of orthogonal matrices, more specifically orthogonal matrices with determinant 1 and real coefficients. Since it is a theory of mathematics, I think that it is difficult to understand if are not familiar with mathematics.

However, I don't think it's a good idea for a contributor who isn't familiar with mathematics to make changes that aren't good for users. This is true for any function, not just this one. The reason I thought about changing the specifications of orbitControl() was because I was concerned about that happening.

I also thought using quaternions would work, but gave up for two reasons. One is that p5.js doesn't introduce quaternions, and the other is that I realized that if I did, it would only make the implementation more difficult.

In particular, when the coefficient comparison method introduced earlier was introduced, the execution speed doubled. So I think the direction is correct.

Most of all, that simple slerp() approach doesn't work. p5.Vector.slerp() is expected to play a big role in the creativeCoding, so even if it can't be used for camera interpolation, it's useful enough.

The code may be a little difficult to understand, but could you adopt it in this format?

[inaridarkfox4231]

The implementation is not without problems. There is a problem of arbitrariness.

In this implementation, the center is linearly interpolated, then the new front vector is found by interpolating the orthonormal basis, and the new eye position is calculated. This method is based on p5.EasyCam. It uses this to reset orbitControl(), but orbitControl() doesn't move the center, so it goes well with this method. However, it is not compatible with methods such as pan and tilt that move the center. The position of the eye will move slightly. In that case, a method that linearly interpolates the eye position and calculates the center position based on that would be more suitable. Alternatively, there may be a way to linearly interpolate the centroids of the eye and center.

However, I can't implement this and that, so this time I simply adopted the method of linearly interpolating the center, which is fit with orbitControl() well. Maybe it would be nice to have it as options, but this implementation doesn't spoil the appearance that much, so I think it's fine for this time.

[inaridarkfox4231]

Perhaps, if there is a mathematical operation to find the ratio of position between the eyes and the center that is natural in any case, orbitControl(), pan(), or tilt() may be interpolated naturally...

[inaridarkfox4231]

Although the amount of calculation will increase slightly, I came up with the following method. First, take points between the eyes and the center in equal proportions for each camera. and consider the distance between them. If you calculate the ratio that minimizes this, it will be a reasonable interpolation in both cases... Of course, the case of parallel movement is excluded. All distances are the same regardless of percentage, so we can uniformly take 0. The performance will drop a little, but it's not a function that call many times, so I think it's worth working on...

[inaridarkfox4231]

A little redundant part increased, but I tried to improve it as follows.

p5.Camera.prototype.slerp = function(cam0, cam1, amt){
  // If t is 0 or 1, do not interpolate and set the argument camera.
  if (amt === 0) {
    this.set(cam0);
    return;
  } else if (amt === 1) {
    this.set(cam1);
    return;
  }

  // For this cameras is ortho, assume that cam0 and cam1 are also ortho
  // and interpolate the elements of the projection matrix.
  if (this.projMatrix.mat4[15] !== 0) {
    this.projMatrix.mat4[0] =
      (1 - amt) * cam0.projMatrix.mat4[0] + amt * cam1.projMatrix.mat4[0];
    this.projMatrix.mat4[5] =
      (1 - amt) * cam0.projMatrix.mat4[5] + amt * cam1.projMatrix.mat4[5];
  }
  // If the camera is active, make uPMatrix reflect changes in projMatrix.
  if (this._isActive()) {
    this._renderer.uPMatrix.mat4 = this.projMatrix.mat4.slice();
  }

  // Calculate the distance between eye and center for each camera.
  const dist0 = Math.hypot(
    cam0.eyeX - cam0.centerX,
    cam0.eyeY - cam0.centerY,
    cam0.eyeZ - cam0.centerZ
  );
  const dist1 = Math.hypot(
    cam1.eyeX - cam1.centerX,
    cam1.eyeY - cam1.centerY,
    cam1.eyeZ - cam1.centerZ
  );
  // Then linearly interpolate them by amt.
  const lerpedDist = (1 - amt) * dist0 + amt * dist1;

  // Next, calculate the ratio to interpolate the eye and center by a constant
  // ratio for each camera. This ratio is the same for both. Also, with this ratio
  // of points, the distance is the minimum distance of the two points of
  // the same ratio.
  const eyeDiff = new p5.Vector(
    cam0.eyeX - cam1.eyeX, cam0.eyeY - cam1.eyeY, cam0.eyeZ - cam1.eyeZ
  );
  const diffDiff = new p5.Vector(
    cam0.eyeX - cam1.eyeX - cam0.centerX + cam1.centerX,
    cam0.eyeY - cam1.eyeY - cam0.centerY + cam1.centerY,
    cam0.eyeZ - cam1.eyeZ - cam0.centerZ + cam1.centerZ
  );
  const divider = diffDiff.magSq();
  let ratio = 1; // default.
  if (divider > 0.000001){
    ratio = p5.Vector.dot(eyeDiff, diffDiff) / divider;
    ratio = Math.max(0, Math.min(ratio, 1));
  }

  // Take the appropriate proportions and work out the points
  // that are between the new viewpoint and the new center position.
  const medium0X = (1-ratio) * cam0.eyeX + ratio * cam0.centerX;
  const medium0Y = (1-ratio) * cam0.eyeY + ratio * cam0.centerY;
  const medium0Z = (1-ratio) * cam0.eyeZ + ratio * cam0.centerZ;

  const medium1X = (1-ratio) * cam1.eyeX + ratio * cam1.centerX;
  const medium1Y = (1-ratio) * cam1.eyeY + ratio * cam1.centerY;
  const medium1Z = (1-ratio) * cam1.eyeZ + ratio * cam1.centerZ;

  const lerpedMediumX = medium0X * (1-amt) + medium1X * amt;
  const lerpedMediumY = medium0Y * (1-amt) + medium1Y * amt;
  const lerpedMediumZ = medium0Z * (1-amt) + medium1Z * amt;

  // Prepare each of rotation matrix from their camera matrix
  const mat0 = cam0.cameraMatrix.mat4;
  const mat1 = cam1.cameraMatrix.mat4;
  const m0 = [
    mat0[0], mat0[1], mat0[2],
    mat0[4], mat0[5], mat0[6],
    mat0[8], mat0[9], mat0[10]
  ];
  const m1 = [
    mat1[0], mat1[1], mat1[2],
    mat1[4], mat1[5], mat1[6],
    mat1[8], mat1[9], mat1[10]
  ];
  // Create the inverse matrix of mat0 by transposing mat0,
  // and multiply it to mat1 from the right.
  // This matrix represents the difference between the two.
  const m = [
    m1[0] * m0[0] + m1[1] * m0[1] + m1[2] * m0[2],
    m1[0] * m0[3] + m1[1] * m0[4] + m1[2] * m0[5],
    m1[0] * m0[6] + m1[1] * m0[7] + m1[2] * m0[8],

    m1[3] * m0[0] + m1[4] * m0[1] + m1[5] * m0[2],
    m1[3] * m0[3] + m1[4] * m0[4] + m1[5] * m0[5],
    m1[3] * m0[6] + m1[4] * m0[7] + m1[5] * m0[8],

    m1[6] * m0[0] + m1[7] * m0[1] + m1[8] * m0[2],
    m1[6] * m0[3] + m1[7] * m0[4] + m1[8] * m0[5],
    m1[6] * m0[6] + m1[7] * m0[7] + m1[8] * m0[8]
  ];

  // Calculate the trace and from it the cos value of the angle.
  // trace = 1 + 2 * cosTheta.
  let cosTheta = 0.5 * (m[0] + m[4] + m[8] - 1);
  // If the angle is close to 0, the two matrices are very close,
  // so in that case we execute linearly interpolate.
  if (1 - cosTheta < 0.0000001) {
    // Obtain the front vector and up vector by linear interpolation
    // and normalize them.
    const lerpedFront = new p5.Vector(
      (1 - amt) * m0[2] + amt * m1[2],
      (1 - amt) * m0[5] + amt * m1[5],
      (1 - amt) * m0[8] + amt * m1[8]
    ).normalize();
    const lerpedUp = new p5.Vector(
      (1 - amt) * m0[1] + amt * m1[1],
      (1 - amt) * m0[4] + amt * m1[4],
      (1 - amt) * m0[7] + amt * m1[7]
    ).normalize();

    // set the camera
    // The eye position and center position are calculated based on the front vector.
    this.camera(
      lerpedMediumX + ratio * lerpedDist * lerpedFront.x,
      lerpedMediumY + ratio * lerpedDist * lerpedFront.y,
      lerpedMediumZ + ratio * lerpedDist * lerpedFront.z,
      lerpedMediumX + (ratio-1) * lerpedDist * lerpedFront.x,
      lerpedMediumY + (ratio-1) * lerpedDist * lerpedFront.y,
      lerpedMediumZ + (ratio-1) * lerpedDist * lerpedFront.z,
      lerpedUp.x, lerpedUp.y, lerpedUp.z
    );
    return;
  }

  // Calculates the axis vector and the angle of the difference orthogonal matrix.
  // similar calculation is here:
  // https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294
  let a, b, c, sinTheta;
  let invOneMinusCosTheta = 1 / (1 - cosTheta);
  if (m[0] > m[4] && m[0] > m[8]) {
    a = Math.sqrt((m[0] - cosTheta) * invOneMinusCosTheta); // not zero.
    invOneMinusCosTheta /= a;
    b = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
    c = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
    sinTheta = 0.5 * (m[7] - m[5]) / a;
  } else if (m[4] > m[8]) {
    b = Math.sqrt((m[4] - cosTheta) * invOneMinusCosTheta); // not zero.
    invOneMinusCosTheta /= b;
    c = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
    a = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
    sinTheta = 0.5 * (m[2] - m[6]) / b;
  } else {
    c = Math.sqrt((m[8] - cosTheta) * invOneMinusCosTheta); // not zero.
    invOneMinusCosTheta /= c;
    a = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
    b = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
    sinTheta = 0.5 * (m[3] - m[1]) / c;
  }

  // Constructs a new matrix after interpolating the angles.
  // Multiplying mat0 by the first matrix yields mat1, but by creating a state
  // in the middle of that matrix, you can obtain a matrix that is
  // an intermediate state between mat0 and mat1.
  const angle = amt * Math.atan2(sinTheta, cosTheta);
  const cosAngle = Math.cos(angle);
  const sinAngle = Math.sin(angle);
  const oneMinusCosAngle = 1 - cosAngle;
  const ab = a * b;
  const bc = b * c;
  const ca = c * a;
  const result = [
    cosAngle + oneMinusCosAngle * a * a,
    oneMinusCosAngle * ab - sinAngle * c,
    oneMinusCosAngle * ca + sinAngle * b,
    oneMinusCosAngle * ab + sinAngle * c,
    cosAngle + oneMinusCosAngle * b * b,
    oneMinusCosAngle * bc - sinAngle * a,
    oneMinusCosAngle * ca - sinAngle * b,
    oneMinusCosAngle * bc + sinAngle * a,
    cosAngle + oneMinusCosAngle * c * c
  ];

  // Multiply this to mat0 from left to get the interpolated front vector.
  //const newFront = new p5.Vector(
  const newFrontX = result[0] * m0[2] + result[1] * m0[5] + result[2] * m0[8];
  const newFrontY = result[3] * m0[2] + result[4] * m0[5] + result[5] * m0[8];
  const newFrontZ = result[6] * m0[2] + result[7] * m0[5] + result[8] * m0[8];
 // );

  // We also get the up vector in the same way and set the camera.
  // The eye position and center position are calculated based on the front vector.
  this.camera(
    lerpedMediumX + ratio * lerpedDist * newFrontX,
    lerpedMediumY + ratio * lerpedDist * newFrontY,
    lerpedMediumZ + ratio * lerpedDist * newFrontZ,
    lerpedMediumX + (ratio-1) * lerpedDist * newFrontX,
    lerpedMediumY + (ratio-1) * lerpedDist * newFrontY,
    lerpedMediumZ + (ratio-1) * lerpedDist * newFrontZ,
    result[0] * m0[1] + result[1] * m0[4] + result[2] * m0[7],
    result[3] * m0[1] + result[4] * m0[4] + result[5] * m0[7],
    result[6] * m0[1] + result[7] * m0[4] + result[8] * m0[7]
  );
}

This logic calculates a ratio instead of simply linearly interpolating the center position. Take a point between the viewpoint and the center for each camera in that ratio. Then we take those points interpolated by amt and determine the new viewpoint and center position from that and the new front vector. The ratio is the shortest distance when considering the line segment connecting the viewpoint and the center for each camera. Calculate the ratio of the shortest distance when points are taken at the same ratio.

With this method, if the viewpoint is fixed, linear interpolation is performed at the viewpoint, and if the center is fixed, linear interpolation is performed at the center, resulting in reasonable interpolation. If both move, the point halfway between them is taken.

To speed up calculations, only the components are calculated if vectors need not be created. The processing to create a vector is heavy, so I reduced it as much as possible. As a result, we were able to achieve almost the same performance as before the improvement.

For example, if cam1 is cam0 panned by 0.8PI, the resulting camera from slerp() by amount 3/8 will be cam0 panned by 0.3PI. Tilt and _orbitFree have similar results.

More verbose, but I think it's a better interpolation...

[davepagurek]

See what happens when you rotate 180 degrees vertically.

Makes sense, thanks for finding that example!

With this method, if the viewpoint is fixed, linear interpolation is performed at the viewpoint, and if the center is fixed, linear interpolation is performed at the center, resulting in reasonable interpolation. If both move, the point halfway between them is taken.

This is a great property! Let's also add this in a comment in the code to help future readers understand the motivation for this method. (Maybe also a unit test showing the linear behaviour with a fixed center and with a fixed viewpoint?)

[inaridarkfox4231]

OK, I think it's easier to understand! I know it's hard to read this code, so I'm open to any suggestions to rewrite it for readability or comments. I've already checked the unit test. Rather, I thought that it would be easier if this property was present in the process of creating unit tests. I'm planning to run the unit test with pan(), tilt(), _orbit(), _orbitFree(). I think that's the easiest way to confirm. Similar to p5.Vector.slerp(), it also checks for out-of-bounds cases.