Closed inaridarkfox4231 closed 1 year ago
The first example may be easier to understand using pan(). I think I will rewrite it.
I decided to rewrite setup() in the first example as follows:
function setup() {
createCanvas(100, 100, WEBGL);
strokeWeight(3);
// camera for slerp.
cam = createCamera();
// cam0 is looking at the cube from the front.
// cam1 is pointing straight to the right in the cube
// at the same position as cam0 by doing a pan(-PI/2).
cam0 = createCamera();
cam1 = createCamera();
cam1.pan(-PI/2);
// we only use cam.
setCamera(cam);
}
I thought this was easier to understand than specifying it with the camera() function.
I tried to execute it with tilt(), but I gave up because it broke the camera when tilt(±PI/2) is executed because of the specification that does not rewrite the up vector. I can't decide at this time if this should be fixed, so I'm putting it on hold. Even if it should be addressed, it is not covered in this issue. I think that even if the up vector is also rotated at the same time, the problem will not occur, but I don't know. If I deal with it, I think I'll create a new issue.
Specifically, if you do this in the example of tilt(), the camera will not return.
https://github.com/processing/p5.js/assets/39549290/4ee7a2dd-8230-4f39-9498-72a959611f08
If tilt() is specified so that the up vector changes, the behavior of pan() after tilt() will change. I don't know if this is good or bad, so I'm not going to touch on this issue.
I will add the results of the benchmark test instead.
benchmark test: 12th Gen Intel(R) Core(TM) i7-1260P 2.10 GHz Windows 11 Home
After 60 frames with 50000 calls per frame and taking the average, the processing speed did not drop at all. Actually, I think it's a few times at most, but I was able to confirm that it was fast enough.
I think this is a useful function to add!
I wonder if we can leverage the slerp code you added to p5.Vector to reduce the code complexity in p5.Camera and make it easier for future contributors to see what's going on? e.g. doing something like this:
const eye0 = createVector(cam0.eyeX, cam0.eyeY, cam0.eyeZ);
const eye1 = createVector(cam1.eyeX, cam1.eyeY, cam1.eyeZ);
const center0 = createVector(cam0.centerX, cam0.centerY, cam0.centerZ);
const center1 = createVector(cam1.centerX, cam1.centerY, cam1.centerZ);
const up0 = createVector(cam0.upX, cam0.upY, cam0.upZ);
const up1 = createVector(cam1.upX, cam1.upY, cam1.upZ);
const newCenter = center0.copy().lerp(center1, amt);
const newLook = center0.copy().sub(eye0).slerp(center1.copy().sub(eye1), amt);
const newEye = newCenter.copy().sub(newLook);
const newUp = up0.copy().slerp(up1, amt);
this.camera(
newEye.x, newEye.y, newEye.z,
newCenter.x, newCenter.y, newCenter.z,
newUp.x, newUp.y, newUp.z
);
Quick copy of your sketch with this code pasted in: https://editor.p5js.org/davepagurek/sketches/wkLdQ9DUT
I haven't thought through all the implications of this, so if there are cases that the matrix math handles that this doesn't, there's nothing wrong with using the version you suggested.
Unfortunately, I cannot accept that proposal. See what happens when you rotate 180 degrees vertically.
https://github.com/processing/p5.js/assets/39549290/ec977c09-3c4b-42f8-9625-53edf3189fb2
I wanted to introduce vector slerp() either purely out of curiosity or convenience. Not for cameras. At first, I was confused between vector slerp and camera slerp, or more precisely, quaternion slerp. But the two are actually very different. Even if only the vector is interpolated with slerp(), the intermediate process will not become an orthonormal basis, so the interpolation will be messy. So it doesn't work. This proposal is useless, sorry.
I think the next part is probably the hardest to understand.
// Calculates the axis vector and the angle of the difference orthogonal matrix.
// similar calculation is here:
// https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294
let a, b, c, sinTheta;
let invOneMinusCosTheta = 1 / (1 - cosTheta);
if (m[0] > m[4] && m[0] > m[8]) {
a = Math.sqrt((m[0] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= a;
b = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
c = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[7] - m[5]) / a;
} else if (m[4] > m[8]) {
b = Math.sqrt((m[4] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= b;
c = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
a = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[2] - m[6]) / b;
} else {
c = Math.sqrt((m[8] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= c;
a = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
b = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[3] - m[1]) / c;
}
What we're doing here is that we're assuming some form of orthogonal matrix, and we're finding the variables that appear in it by comparing the coefficients. You'll see what it looks like in the next part.
const angle = amt * Math.atan2(sinTheta, cosTheta);
const cosAngle = Math.cos(angle);
const sinAngle = Math.sin(angle);
const oneMinusCosAngle = 1 - cosAngle;
const ab = a * b;
const bc = b * c;
const ca = c * a;
const result = [
cosAngle + oneMinusCosAngle * a * a,
oneMinusCosAngle * ab - sinAngle * c,
oneMinusCosAngle * ca + sinAngle * b,
oneMinusCosAngle * ab + sinAngle * c,
cosAngle + oneMinusCosAngle * b * b,
oneMinusCosAngle * bc - sinAngle * a,
oneMinusCosAngle * ca - sinAngle * b,
oneMinusCosAngle * bc + sinAngle * a,
cosAngle + oneMinusCosAngle * c * c
];
I'm looking for a comparison with this. This is due to the nature of orthogonal matrices, more specifically orthogonal matrices with determinant 1 and real coefficients. Since it is a theory of mathematics, I think that it is difficult to understand if are not familiar with mathematics.
However, I don't think it's a good idea for a contributor who isn't familiar with mathematics to make changes that aren't good for users. This is true for any function, not just this one. The reason I thought about changing the specifications of orbitControl() was because I was concerned about that happening.
I also thought using quaternions would work, but gave up for two reasons. One is that p5.js doesn't introduce quaternions, and the other is that I realized that if I did, it would only make the implementation more difficult.
In particular, when the coefficient comparison method introduced earlier was introduced, the execution speed doubled. So I think the direction is correct.
Most of all, that simple slerp() approach doesn't work. p5.Vector.slerp() is expected to play a big role in the creativeCoding, so even if it can't be used for camera interpolation, it's useful enough.
The code may be a little difficult to understand, but could you adopt it in this format?
The implementation is not without problems. There is a problem of arbitrariness.
In this implementation, the center is linearly interpolated, then the new front vector is found by interpolating the orthonormal basis, and the new eye position is calculated. This method is based on p5.EasyCam. It uses this to reset orbitControl(), but orbitControl() doesn't move the center, so it goes well with this method. However, it is not compatible with methods such as pan and tilt that move the center. The position of the eye will move slightly. In that case, a method that linearly interpolates the eye position and calculates the center position based on that would be more suitable. Alternatively, there may be a way to linearly interpolate the centroids of the eye and center.
However, I can't implement this and that, so this time I simply adopted the method of linearly interpolating the center, which is fit with orbitControl() well. Maybe it would be nice to have it as options, but this implementation doesn't spoil the appearance that much, so I think it's fine for this time.
Perhaps, if there is a mathematical operation to find the ratio of position between the eyes and the center that is natural in any case, orbitControl(), pan(), or tilt() may be interpolated naturally...
Although the amount of calculation will increase slightly, I came up with the following method. First, take points between the eyes and the center in equal proportions for each camera. and consider the distance between them. If you calculate the ratio that minimizes this, it will be a reasonable interpolation in both cases... Of course, the case of parallel movement is excluded. All distances are the same regardless of percentage, so we can uniformly take 0. The performance will drop a little, but it's not a function that call many times, so I think it's worth working on...
A little redundant part increased, but I tried to improve it as follows.
p5.Camera.prototype.slerp = function(cam0, cam1, amt){
// If t is 0 or 1, do not interpolate and set the argument camera.
if (amt === 0) {
this.set(cam0);
return;
} else if (amt === 1) {
this.set(cam1);
return;
}
// For this cameras is ortho, assume that cam0 and cam1 are also ortho
// and interpolate the elements of the projection matrix.
if (this.projMatrix.mat4[15] !== 0) {
this.projMatrix.mat4[0] =
(1 - amt) * cam0.projMatrix.mat4[0] + amt * cam1.projMatrix.mat4[0];
this.projMatrix.mat4[5] =
(1 - amt) * cam0.projMatrix.mat4[5] + amt * cam1.projMatrix.mat4[5];
}
// If the camera is active, make uPMatrix reflect changes in projMatrix.
if (this._isActive()) {
this._renderer.uPMatrix.mat4 = this.projMatrix.mat4.slice();
}
// Calculate the distance between eye and center for each camera.
const dist0 = Math.hypot(
cam0.eyeX - cam0.centerX,
cam0.eyeY - cam0.centerY,
cam0.eyeZ - cam0.centerZ
);
const dist1 = Math.hypot(
cam1.eyeX - cam1.centerX,
cam1.eyeY - cam1.centerY,
cam1.eyeZ - cam1.centerZ
);
// Then linearly interpolate them by amt.
const lerpedDist = (1 - amt) * dist0 + amt * dist1;
// Next, calculate the ratio to interpolate the eye and center by a constant
// ratio for each camera. This ratio is the same for both. Also, with this ratio
// of points, the distance is the minimum distance of the two points of
// the same ratio.
const eyeDiff = new p5.Vector(
cam0.eyeX - cam1.eyeX, cam0.eyeY - cam1.eyeY, cam0.eyeZ - cam1.eyeZ
);
const diffDiff = new p5.Vector(
cam0.eyeX - cam1.eyeX - cam0.centerX + cam1.centerX,
cam0.eyeY - cam1.eyeY - cam0.centerY + cam1.centerY,
cam0.eyeZ - cam1.eyeZ - cam0.centerZ + cam1.centerZ
);
const divider = diffDiff.magSq();
let ratio = 1; // default.
if (divider > 0.000001){
ratio = p5.Vector.dot(eyeDiff, diffDiff) / divider;
ratio = Math.max(0, Math.min(ratio, 1));
}
// Take the appropriate proportions and work out the points
// that are between the new viewpoint and the new center position.
const medium0X = (1-ratio) * cam0.eyeX + ratio * cam0.centerX;
const medium0Y = (1-ratio) * cam0.eyeY + ratio * cam0.centerY;
const medium0Z = (1-ratio) * cam0.eyeZ + ratio * cam0.centerZ;
const medium1X = (1-ratio) * cam1.eyeX + ratio * cam1.centerX;
const medium1Y = (1-ratio) * cam1.eyeY + ratio * cam1.centerY;
const medium1Z = (1-ratio) * cam1.eyeZ + ratio * cam1.centerZ;
const lerpedMediumX = medium0X * (1-amt) + medium1X * amt;
const lerpedMediumY = medium0Y * (1-amt) + medium1Y * amt;
const lerpedMediumZ = medium0Z * (1-amt) + medium1Z * amt;
// Prepare each of rotation matrix from their camera matrix
const mat0 = cam0.cameraMatrix.mat4;
const mat1 = cam1.cameraMatrix.mat4;
const m0 = [
mat0[0], mat0[1], mat0[2],
mat0[4], mat0[5], mat0[6],
mat0[8], mat0[9], mat0[10]
];
const m1 = [
mat1[0], mat1[1], mat1[2],
mat1[4], mat1[5], mat1[6],
mat1[8], mat1[9], mat1[10]
];
// Create the inverse matrix of mat0 by transposing mat0,
// and multiply it to mat1 from the right.
// This matrix represents the difference between the two.
const m = [
m1[0] * m0[0] + m1[1] * m0[1] + m1[2] * m0[2],
m1[0] * m0[3] + m1[1] * m0[4] + m1[2] * m0[5],
m1[0] * m0[6] + m1[1] * m0[7] + m1[2] * m0[8],
m1[3] * m0[0] + m1[4] * m0[1] + m1[5] * m0[2],
m1[3] * m0[3] + m1[4] * m0[4] + m1[5] * m0[5],
m1[3] * m0[6] + m1[4] * m0[7] + m1[5] * m0[8],
m1[6] * m0[0] + m1[7] * m0[1] + m1[8] * m0[2],
m1[6] * m0[3] + m1[7] * m0[4] + m1[8] * m0[5],
m1[6] * m0[6] + m1[7] * m0[7] + m1[8] * m0[8]
];
// Calculate the trace and from it the cos value of the angle.
// trace = 1 + 2 * cosTheta.
let cosTheta = 0.5 * (m[0] + m[4] + m[8] - 1);
// If the angle is close to 0, the two matrices are very close,
// so in that case we execute linearly interpolate.
if (1 - cosTheta < 0.0000001) {
// Obtain the front vector and up vector by linear interpolation
// and normalize them.
const lerpedFront = new p5.Vector(
(1 - amt) * m0[2] + amt * m1[2],
(1 - amt) * m0[5] + amt * m1[5],
(1 - amt) * m0[8] + amt * m1[8]
).normalize();
const lerpedUp = new p5.Vector(
(1 - amt) * m0[1] + amt * m1[1],
(1 - amt) * m0[4] + amt * m1[4],
(1 - amt) * m0[7] + amt * m1[7]
).normalize();
// set the camera
// The eye position and center position are calculated based on the front vector.
this.camera(
lerpedMediumX + ratio * lerpedDist * lerpedFront.x,
lerpedMediumY + ratio * lerpedDist * lerpedFront.y,
lerpedMediumZ + ratio * lerpedDist * lerpedFront.z,
lerpedMediumX + (ratio-1) * lerpedDist * lerpedFront.x,
lerpedMediumY + (ratio-1) * lerpedDist * lerpedFront.y,
lerpedMediumZ + (ratio-1) * lerpedDist * lerpedFront.z,
lerpedUp.x, lerpedUp.y, lerpedUp.z
);
return;
}
// Calculates the axis vector and the angle of the difference orthogonal matrix.
// similar calculation is here:
// https://github.com/mrdoob/three.js/blob/dev/src/math/Quaternion.js#L294
let a, b, c, sinTheta;
let invOneMinusCosTheta = 1 / (1 - cosTheta);
if (m[0] > m[4] && m[0] > m[8]) {
a = Math.sqrt((m[0] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= a;
b = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
c = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[7] - m[5]) / a;
} else if (m[4] > m[8]) {
b = Math.sqrt((m[4] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= b;
c = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
a = 0.5 * (m[1] + m[3]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[2] - m[6]) / b;
} else {
c = Math.sqrt((m[8] - cosTheta) * invOneMinusCosTheta); // not zero.
invOneMinusCosTheta /= c;
a = 0.5 * (m[2] + m[6]) * invOneMinusCosTheta;
b = 0.5 * (m[5] + m[7]) * invOneMinusCosTheta;
sinTheta = 0.5 * (m[3] - m[1]) / c;
}
// Constructs a new matrix after interpolating the angles.
// Multiplying mat0 by the first matrix yields mat1, but by creating a state
// in the middle of that matrix, you can obtain a matrix that is
// an intermediate state between mat0 and mat1.
const angle = amt * Math.atan2(sinTheta, cosTheta);
const cosAngle = Math.cos(angle);
const sinAngle = Math.sin(angle);
const oneMinusCosAngle = 1 - cosAngle;
const ab = a * b;
const bc = b * c;
const ca = c * a;
const result = [
cosAngle + oneMinusCosAngle * a * a,
oneMinusCosAngle * ab - sinAngle * c,
oneMinusCosAngle * ca + sinAngle * b,
oneMinusCosAngle * ab + sinAngle * c,
cosAngle + oneMinusCosAngle * b * b,
oneMinusCosAngle * bc - sinAngle * a,
oneMinusCosAngle * ca - sinAngle * b,
oneMinusCosAngle * bc + sinAngle * a,
cosAngle + oneMinusCosAngle * c * c
];
// Multiply this to mat0 from left to get the interpolated front vector.
//const newFront = new p5.Vector(
const newFrontX = result[0] * m0[2] + result[1] * m0[5] + result[2] * m0[8];
const newFrontY = result[3] * m0[2] + result[4] * m0[5] + result[5] * m0[8];
const newFrontZ = result[6] * m0[2] + result[7] * m0[5] + result[8] * m0[8];
// );
// We also get the up vector in the same way and set the camera.
// The eye position and center position are calculated based on the front vector.
this.camera(
lerpedMediumX + ratio * lerpedDist * newFrontX,
lerpedMediumY + ratio * lerpedDist * newFrontY,
lerpedMediumZ + ratio * lerpedDist * newFrontZ,
lerpedMediumX + (ratio-1) * lerpedDist * newFrontX,
lerpedMediumY + (ratio-1) * lerpedDist * newFrontY,
lerpedMediumZ + (ratio-1) * lerpedDist * newFrontZ,
result[0] * m0[1] + result[1] * m0[4] + result[2] * m0[7],
result[3] * m0[1] + result[4] * m0[4] + result[5] * m0[7],
result[6] * m0[1] + result[7] * m0[4] + result[8] * m0[7]
);
}
This logic calculates a ratio instead of simply linearly interpolating the center position. Take a point between the viewpoint and the center for each camera in that ratio. Then we take those points interpolated by amt and determine the new viewpoint and center position from that and the new front vector. The ratio is the shortest distance when considering the line segment connecting the viewpoint and the center for each camera. Calculate the ratio of the shortest distance when points are taken at the same ratio.
With this method, if the viewpoint is fixed, linear interpolation is performed at the viewpoint, and if the center is fixed, linear interpolation is performed at the center, resulting in reasonable interpolation. If both move, the point halfway between them is taken.
To speed up calculations, only the components are calculated if vectors need not be created. The processing to create a vector is heavy, so I reduced it as much as possible. As a result, we were able to achieve almost the same performance as before the improvement.
For example, if cam1 is cam0 panned by 0.8PI, the resulting camera from slerp() by amount 3/8 will be cam0 panned by 0.3PI. Tilt and _orbitFree have similar results.
More verbose, but I think it's a better interpolation...
See what happens when you rotate 180 degrees vertically.
Makes sense, thanks for finding that example!
With this method, if the viewpoint is fixed, linear interpolation is performed at the viewpoint, and if the center is fixed, linear interpolation is performed at the center, resulting in reasonable interpolation. If both move, the point halfway between them is taken.
This is a great property! Let's also add this in a comment in the code to help future readers understand the motivation for this method. (Maybe also a unit test showing the linear behaviour with a fixed center and with a fixed viewpoint?)
OK, I think it's easier to understand! I know it's hard to read this code, so I'm open to any suggestions to rewrite it for readability or comments. I've already checked the unit test. Rather, I thought that it would be easier if this property was present in the process of creating unit tests. I'm planning to run the unit test with pan(), tilt(), _orbit(), _orbitFree(). I think that's the easiest way to confirm. Similar to p5.Vector.slerp(), it also checks for out-of-bounds cases.
Increasing Access
If there are two cameras, I'd like to implement a function to interpolate each camera's view. For example, if one camera is facing the front and the other is facing the right, the interpolated camera will be facing diagonally to the right. It's easier to understand with an example, so I'll give you a few examples. camera.slerp()_sample1
https://github.com/processing/p5.js/assets/39549290/5af19d8c-9251-40c0-84b1-621364100a66
cam0 is looking straight ahead at the cube from above. On the other hand, cam1 is at the same position as cam0 and facing straight to the right of the cube. Interpolating these between 0 and 1 and setting them to cam will smoothly interpolate the view between them. Only cam is active. slerp() will update uMV and uP if the target is active, so there is no need to call setCamera(). I had set() implemented to achieve this.
Another example concerns orbitControl(). camera.slerp()_sample2
https://github.com/processing/p5.js/assets/39549290/f6ceb617-0649-48d2-aa40-ca7d4922f5a8
Prepare two cameras, initialCam and lastCam. Set the initial state in initialCam. lastCam is used to record the current state of cam when resetting the camera. Double-clicking resets the counter to 0, and until it reaches a constant value, interpolates the view from the initial state and sets it to the camera. Also in this case, no setCamera() is needed as only cam is active. The nice thing about slerp() is that you can chain multiple views without explicitly switching cameras.
Also, some fixes for orbitControl().
We rewrite uP internally so that it can be scaled with ortho(), but this not change the camera's projMatrix. Therefore, first rewrite projMatrix and rewrite it so that its contents are reflected in uP.
If the value of mouseWheelDeltaY is used as it is, the behavior may change due to the difference in value depending on the model. So, I decided to use a constant which multiplied the sign of mouseWheelDeltaY. Behavior remained almost unchanged. I think there is no problem.
Also, I fixed several places where I could write in the global variable cam.
Most appropriate sub-area of p5.js?
Feature request details
The implementation of slerp() is as follows:
It takes two cameras and interpolates their views. If the target camera is active, it will be reflected in uMV and uP and the screen will be updated. All camera projection modes must match. For example, you cannot interpolate a perspective camera and an ortho camera. The projection matrix must also match, but only in the case of ortho(), interpolation is possible if only the 0th or 5th component differs due to orbitControl(). If amt is 0 or 1, the argument camera is set as is.
It looks like there are a lot of redundant calculations involving 3x3 matrices, but the original code had a lot more, so I considered making them into methods. However, in the process of simplification and speedup of the method, it was reduced and almost disappeared. Therefore, it was decided that it would be better to write directly rather than making it a method, but I don't know if this is okay.
In addition to these, I plan to implement the orbitControl() fix I mentioned above.