EXR Metadata Rotation Bug

[adrianmeyerart]

Hi, working on using the new additional passes and the metadata. Found out that there is a bug in the rotation Metadata written into the EXRs.

To get it right I had to do the following: rotate_x = 90 - float(metadata("camera_rotation.y")) rotate_y = float(metadata("camera_rotation.z")) rotate_z = float(metadata("camera_rotation.x"))

Might be cleaner if you would do this directly and write the correct values into the exr, so not everybody has to figure this out by itself.

Position Metadata is correct! Let me know when this is done so I will undo the correction in my code.

Thanks

[buddhi1980]

Corrected. Check if rotation axes are correct now.

[adrianmeyerart]

Thanks, I'll wait for the next Travis CI Build then right?

[adrianmeyerart]

Unfortunately I have discovered another thing...

1. The "stereo_eye_distance" meta value should be multiplied by 2. It seems to be a distance from the stereo center, but not the actual distance between the eyes. My recreated stereo looked correct when using value x 2.

2. Somehow the camera rotation values in the metadata are not exactly correct. After the corrections I made (that you have now included) the camera lines up pretty well. But not extact, (the position seems to exact though I think). There is some kind of rotation offset, its just about 1° of error, but this is a big problem for what I want to achieve. Might also be something with Rotation Orders etc. I have tried all possible rotation orders, but never got it right. (So the best I got was corrections mentioned above and using these values with Rotation Order RX/RY/RZ ) Could you maybe check on this and see if there might be a little error in how you calculate the rotation values? Or the other way around, I could alos try to not use the rotation values but line up the camera using main_camera_target values and main_camera_top_xyz. But I think it might be cleaner and also less error-prone if the correct rotation values come from MDB.

Thanks so much!

[adrianmeyerart]

PS. When this works it will be very easy with the tool I build to combine mandelbulber renderings with other normal 3D Objects/Renderings. Like real Volumetric Clouds etc.

[adrianmeyerart]

... Probably it would also be great to export the metadata with a bit more precision. Example:

EXR meta: exr/camera.x : 0.544565 Mandelbulber value: main_camera_x : 0,5445646302125389

[buddhi1980]

I have already increased accuracy of coordinates in this commit https://github.com/buddhi1980/mandelbulber2/commit/3fd4372dc3f076df384718112e5f4b9bdef26e3c

[adrianmeyerart]

I have already increased accuracy of coordinates in this commit 3fd4372

Nice! Unfortunately this won't fix the rotation problem. This is more than precision problem...

PS. adding main_camera_top_xyz to meta would be nice anyways. even though as I mentioned rotation values should be fixed internally

[buddhi1980]

The problem is probably in the sequence of rotations. How they are defined in Houdini?

[buddhi1980]

You have to remember that in Mandelbulber axes are swapped. y is depth and z is vertical. The reason of this is to be complain with fractal coordinates (like in M-Set). XY plane is defined like for flat fractal, and z is extension for third dimension. Default view is from side of the 3D fractal.

Rotation matrix is defined in following way:

    // preparing rotation matrix
    mRot.RotateZ(viewAngle.x); // yaw
    mRot.RotateX(viewAngle.y); // pitch
    mRot.RotateY(viewAngle.z); // roll

void CRotationMatrix::RotateX(double angle)
{
    if (angle != 0.0)
    {
        CMatrix33 rot;
        const double s = sin(angle);
        const double c = cos(angle);
        rot.m11 = 1.0;
        rot.m22 = c;
        rot.m33 = c;
        rot.m23 = -s;
        rot.m32 = s;
        matrix = matrix * rot;
        zero = false;
    }
}

void CRotationMatrix::RotateY(double angle)
{
    if (angle != 0.0)
    {
        CMatrix33 rot;
        const double s = sin(angle);
        const double c = cos(angle);
        rot.m22 = 1.0;
        rot.m33 = c;
        rot.m11 = c;
        rot.m31 = -s;
        rot.m13 = s;
        matrix = matrix * rot;
        zero = false;
    }
}

void CRotationMatrix::RotateZ(double angle)
{
    if (angle != 0.0)
    {
        CMatrix33 rot;
        const double s = sin(angle);
        const double c = cos(angle);
        rot.m33 = 1.0;
        rot.m11 = c;
        rot.m22 = c;
        rot.m12 = -s;
        rot.m21 = s;
        matrix = matrix * rot;
        zero = false;
    }
}

[adrianmeyerart]

The problem is probably in the sequence of rotations. How they are defined in Houdini?

Might be yes... In Houdini you can choose the Rotation Order (All possible ones). But the default is: RX/RY/RZ Transform Order: Scale/Rotation/Translate

Coordinate System: Y-Up Z-Forward

[adrianmeyerart]

But I am not converting the coordinate system. I just take the values raw as they are (despite the rotation corrections I mentioned). So more precisely I use the raw position values from the EXR word position image layer and directly convert each pixel into s 3D Point. Also I directly import the raw Camera position values from Metadata. Now the fractal Pointcloud scene has an unchanged orientation as in Mandelbulber. So Y stays Y, Z stays Z etc. Just the scene does not line up with Houdini’s default orientation, but that does not matter. But to achieve that also for the camera orientation Y stays Y, Z stays Z and X stays X I had to play around and found that I had to do the rotation corrections mentioned above. Now the scenes in Houdini and Mandelbulber line up identical. Despite that little rotation mismatch that remains somehow and causes problems.

Sorry, that was complicated description :-P Hope you know what I mean.

[buddhi1980]

How you convert degrees to radians? Can you show me it in your code?

wt., 15 paź 2019, 11:48 użytkownik Adrian Meyer notifications@github.com napisał:

But I am not converting the coordinate system. I just take the values raw as they are (despite the rotation corrections I mentioned). So more precisely I use the raw position values from the EXR word position image layer and directly convert each pixel into s 3D Point. Also I directly import the raw Camera position values from Metadata. Now the fractal Pointcloud scene has an unchanged orientation as in Mandelbulber. So Y stays Y, Z stays Z etc. Just the scene does not line up with Houdini’s default orientation, but that does not matter. But to achieve that also for the camera orientation Y stays Y, Z stays Z and X stays X I had to play around and found that I had to do the rotation corrections mentioned above. Now the scenes in Houdini and Mandelbulber line up identical. Despite that little rotation mismatch that remains somehow.

Sorry, that was complicated description :-P Hope you know what I mean.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/buddhi1980/mandelbulber2/issues/722?email_source=notifications&email_token=ACZHWXTCTTMQQFCX72H3XP3QOWGXHA5CNFSM4I7YDAKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBIEP2Y#issuecomment-542132203, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZHWXUEVEILWJ54LKYMPXDQOWGXHANCNFSM4I7YDAKA .

[adrianmeyerart]

I don’t convert degrees to radians at all. The rotation values in the metadata are degrees, exactly what I need in Houdini. So no conversion. Just raw values.

[adrianmeyerart]

Rotation matrix is defined in following way:

  // preparing rotation matrix
  mRot.RotateZ(viewAngle.x); // yaw
  mRot.RotateX(viewAngle.y); // pitch
  mRot.RotateY(viewAngle.z); // roll

void CRotationMatrix::RotateX(double angle)
{
  if (angle != 0.0)
  {
      CMatrix33 rot;
      const double s = sin(angle);
      const double c = cos(angle);
      rot.m11 = 1.0;
      rot.m22 = c;
      rot.m33 = c;
      rot.m23 = -s;
      rot.m32 = s;
      matrix = matrix * rot;
      zero = false;
  }
}

void CRotationMatrix::RotateY(double angle)
{
  if (angle != 0.0)
  {
      CMatrix33 rot;
      const double s = sin(angle);
      const double c = cos(angle);
      rot.m22 = 1.0;
      rot.m33 = c;
      rot.m11 = c;
      rot.m31 = -s;
      rot.m13 = s;
      matrix = matrix * rot;
      zero = false;
  }
}

void CRotationMatrix::RotateZ(double angle)
{
  if (angle != 0.0)
  {
      CMatrix33 rot;
      const double s = sin(angle);
      const double c = cos(angle);
      rot.m33 = 1.0;
      rot.m11 = c;
      rot.m22 = c;
      rot.m12 = -s;
      rot.m21 = s;
      matrix = matrix * rot;
      zero = false;
  }
}

Hey, I don't quite get this... Could you send me the link git link so I can see it in context? What is "viewAngle.x" ? Is it "main_camera_top_x" ?

Also I noticed that I can add Yaw/Pitch/Rotation to Keyframes. If I convert this to Flight I have Rotaion Keyframes for every frame. Would this be exactly the same as the Rotation values that you put in the EXR metadata?

[buddhi1980]

In this file there is a code for rotation matrices: https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/src/algebra.cpp

Here is some theory what is rotation matrix: https://en.wikipedia.org/wiki/Rotation_matrix

viewAngle.x, viewAngle.y, viewAngle.z are components of rotation stored in vector structure. viewAngle.x is yaw viewAngle.y is pitch viewAngle.z is roll

main_camera_top is a vector which is perpendicular to the front of the camera (to the line between camera and target) and directed towards top of the camera. E.g. if you change roll angle of the camera, this vector rotates.

Here is preparation of rotation matrix: https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/opencl/engines/fast_engine.cl#L109

        // main rotation matrix
        matrix33 rot;
        rot.m1 = (float3){1.0f, 0.0f, 0.0f};
        rot.m2 = (float3){0.0f, 1.0f, 0.0f};
        rot.m3 = (float3){0.0f, 0.0f, 1.0f};

        rot = RotateZ(rot, consts->params.viewAngle.x);
        rot = RotateX(rot, consts->params.viewAngle.y);
        rot = RotateY(rot, consts->params.viewAngle.z);

Here us usage of rotation matrix https://github.com/buddhi1980/mandelbulber2/blob/master/mandelbulber2/opencl/engines/fast_engine.cl#L158

        float3 viewVectorNotRotated = CalculateViewVector(normalizedScreenPoint, consts->params.fov);
        float3 viewVector = Matrix33MulFloat3(rot, viewVectorNotRotated);

viewVector is a direction of ray starting from the camera. In the center of the screen is {0, 1, 0} (towards y direction which is depth - default view is from side of 3D fractal)

    viewVector.x = normalizedPoint.x * fov;
    viewVector.y = 1.0f;
    viewVector.z = normalizedPoint.y * fov;
    viewVector = normalize(viewVector);

[zebastian]

@buddhi1980 can we close this?

[buddhi1980]

It is still open topic

[adrianmeyerart]

Yes still open...

[adrianmeyerart]

Hey guys, back working on the mandelbulber / houdini / nuke project.

I spend some time to try and resolve this issue. I got closer, but not fully there. I hope we can fix it together. I made a little video where I explain what I did, guess thats much easier to understand. https://vimeo.com/397787998/299692b8c3 (sry something went a bit wrong with recording, some wird artefacts) Hope you find the time to check it out.

Thanks!!

Here the VEX code used:

Values:

from = {0.536337, -0.678756, 6.65716}
to = {-0.0423664, -1.13841,6.52942}
top = {0.0755473, 0.177564, -0.981205}
trans_order = 0 // XFORM_SRT
rot_order = 0   // XFORM_XYZ

// mandelbulber cam data
vector from = chv("from"); //exr meta camera (pos)
vector to = chv("to"); //exr meta target
vector top = chv("top"); //camera_top (from .fract file flightanimation)

// calculate vectors
vector forward = normalize(from - to);
vector right = cross(normalize(top), forward);
vector up = cross(forward, right); 

// prepare xform matrix
vector4 right_c = set(right.x, right.y, right.z, 0);
vector4 up_c = set(up.x, up.y, up.z, 0);
vector4 forward_c = set(forward.x, forward.y, forward.z, 0);
vector4 from_c = set(from.x, from.y, from.z, 1);

// set xform matrix
matrix m = set(right_c, up_c, forward_c, from_c);
printf("\n%s", m);

// extract rotation angles from matrix
int trans_order = chi("trans_order"); // XFORM_SRT
int rot_order = chi("rot_order");     // XFORM_XYZ
vector pivot = {0,0,0};
vector pivot_rotate = pivot;
vector t,r,s,shears;
// cracktransform returns rotation angles in degrees
cracktransform(trans_order, rot_order, pivot, pivot_rotate, m, t, r, s, shears);

// export rotation angles
v@rotation_angles = r;
printf("\n%s", v@rotation_angles);

[mclarekin]

@ adrian out of interest:

• is the 2.25 y fudge factor consistent with other frames/camera angles
• what happens when you repeat the exercise with target (0,0,0), no camera rotations, and looking straight down the y axis camera at (0, -something, 0)

@ sebastian, krzysztof - can we have camera top coordinates added to metadata

On Mon, Mar 16, 2020 at 4:36 AM Adrian Meyer notifications@github.com wrote:

Hey guys, back working on the mandelbulber / houdini / nuke project.

I spend some time to try and resolve this issue. I got closer, but not fully there. I hope we can fix it together. I made a little video where I explain what I did, guess thats much easier to understand. https://vimeo.com/397787998/299692b8c3 (sry something went a bit wrong with recording, some wird artefacts) Hope you find the time to check it out.

Thanks!!

Here the VEX code used:

Values:

from = {0.536337, -0.678756, 6.65716} to = {-0.0423664, -1.13841,6.52942} top = {0.0755473, 0.177564, -0.981205} trans_order = 0 // XFORM_SRT rot_order = 0 // XFORM_XYZ

// mandelbulber cam data vector from = chv("from"); //exr meta camera (pos) vector to = chv("to"); //exr meta target vector top = chv("top"); //camera_top (from .fract file flightanimation)

// calculate vectors vector forward = normalize(from - to); vector right = cross(normalize(top), forward); vector up = cross(forward, right);

// prepare xform matrix vector4 right_c = set(right.x, right.y, right.z, 0); vector4 up_c = set(up.x, up.y, up.z, 0); vector4 forward_c = set(forward.x, forward.y, forward.z, 0); vector4 from_c = set(from.x, from.y, from.z, 1);

// set xform matrix matrix m = set(right_c, up_c, forward_c, from_c); printf("\n%s", m);

// extract rotation angles from matrix int trans_order = chi("trans_order"); // XFORM_SRT int rot_order = chi("rot_order"); // XFORM_XYZ vector pivot = {0,0,0}; vector pivot_rotate = pivot; vector t,r,s,shears; // cracktransform returns rotation angles in degrees cracktransform(trans_order, rot_order, pivot, pivot_rotate, m, t, r, s, shears);

// export rotation angles v@rotation_angles = r; printf("\n%s", v@rotation_angles);

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/buddhi1980/mandelbulber2/issues/722#issuecomment-599226005, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBNSHG5XIOPBUGLLWTCHKLRHTYZFANCNFSM4I7YDAKA .

[mclarekin]

@adrian I have now watched all of your video, so the fudge factor / eyeballing changes probably varies on positions. I have no idea why the metadata rotations are different to the calculated ones I now realize that the camera-top relationship to target & camera, is not at all what i had assumed :)

So I cannot help, other than saying that a standard menger sponge is often good to test things with. As the geometry is all right angles and has known dimensions.

On Tue, Mar 17, 2020 at 4:39 PM carbuncle grim mclarekin@gmail.com wrote:

@ adrian out of interest:

• is the 2.25 y fudge factor consistent with other frames/camera angles
• what happens when you repeat the exercise with target (0,0,0), no camera rotations, and looking straight down the y axis camera at (0, -something, 0)

@ sebastian, krzysztof - can we have camera top coordinates added to metadata

On Mon, Mar 16, 2020 at 4:36 AM Adrian Meyer notifications@github.com wrote:

Hey guys, back working on the mandelbulber / houdini / nuke project.

I spend some time to try and resolve this issue. I got closer, but not fully there. I hope we can fix it together. I made a little video where I explain what I did, guess thats much easier to understand. https://vimeo.com/397787998/299692b8c3 (sry something went a bit wrong with recording, some wird artefacts) Hope you find the time to check it out.

Thanks!!

Here the VEX code used:

Values:

from = {0.536337, -0.678756, 6.65716} to = {-0.0423664, -1.13841,6.52942} top = {0.0755473, 0.177564, -0.981205} trans_order = 0 // XFORM_SRT rot_order = 0 // XFORM_XYZ

// mandelbulber cam data vector from = chv("from"); //exr meta camera (pos) vector to = chv("to"); //exr meta target vector top = chv("top"); //camera_top (from .fract file flightanimation)

// calculate vectors vector forward = normalize(from - to); vector right = cross(normalize(top), forward); vector up = cross(forward, right);

// prepare xform matrix vector4 right_c = set(right.x, right.y, right.z, 0); vector4 up_c = set(up.x, up.y, up.z, 0); vector4 forward_c = set(forward.x, forward.y, forward.z, 0); vector4 from_c = set(from.x, from.y, from.z, 1);

// set xform matrix matrix m = set(right_c, up_c, forward_c, from_c); printf("\n%s", m);

// extract rotation angles from matrix int trans_order = chi("trans_order"); // XFORM_SRT int rot_order = chi("rot_order"); // XFORM_XYZ vector pivot = {0,0,0}; vector pivot_rotate = pivot; vector t,r,s,shears; // cracktransform returns rotation angles in degrees cracktransform(trans_order, rot_order, pivot, pivot_rotate, m, t, r, s, shears);

// export rotation angles v@rotation_angles = r; printf("\n%s", v@rotation_angles);

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/buddhi1980/mandelbulber2/issues/722#issuecomment-599226005, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBNSHG5XIOPBUGLLWTCHKLRHTYZFANCNFSM4I7YDAKA .

[adrianmeyerart]

hey guys, so I think we're finally kind of there. first off, the very strange fudge factor of 2.25° Y was something I oversaw. in my test scene, there was somehow a value of 2.25 in the "Sweet Spot Angle Horizontal". I never noticed that parameter before and still don't exactly know what it does, seems like kind of a post rotation that does not get included in the actual camera transform data? Sorry that I oversaw this.

anyways, just got rid of that now and also tested with simpler scenes such as Menger Sponge. with my self caluclated xform matrix based on _camera, cameratarget and _cameratop now everything works perfectly as expected.

now I again tested all of this manually by grabbing the values from Mandelbulber Flight Anim Keyframe Spreadsheet. So that I can automate that, would be awesome again if you could include "camera_top" in EXR Metadata!

for the actual rotation values you calculate and export to EXR Metadata, somehow this does not work properly. Also tried to directly grab the values from Mandelbulbers Navigation Panel "Yaw, Pitch, Roll", they seem to be the same values initially with the exr Metadata (makes sense). But also when I manually apply these rotation value sin Houdini it does not give correct results.

so I actually have the feeling that your Mandelbulber Rotation values are incorrect. I guess you dont really use them internally as you also use a cam matrix from _camera, cameratarget and _cameratop. so probably the mistake did never appear.

Though, as I mentioned you recently did some modification on the Rotation Metadata based on my suggestion:

rotate_x = 90 - float(metadata("camera_rotation.y")) rotate_y = float(metadata("camera_rotation.z")) rotate_z = float(metadata("camera_rotation.x"))

but somehow something weird happened there and trhe results are very wrong. (with the intial values without your correction and mine applied it was close to the correct result)

So I would suggest, (unless you find that your rotation values are actually completely incorrect and do a complete fix for this) that you revert the rotation metadata to the point before the corrections (i.e. values like they appear in Mandelbulber Navigation Tab).

but that just for cleanliness, as I mentioned, as soon as I have _cameratop in EXR metadata I can get it properly working now.

Thanks a lot! Adrian

[adrianmeyerart]

BTW. it came to my mind that it would also be an option that you directly write the full cam transform matrix to the metadata. 16 float values opposed to 9 (camera, camera_top, camera_target). Guess thats not really a big overhead and then other people wouldn't have to calculate it on their own.

Might be even nicer. But to test it out, it would be great if you could first export all of that, and than I can try what works best. Afterwards you could strip it down again.

thx

[buddhi1980]

In next days I will add top vector to the metadata. From Camera, Target and Top vector you can calculate rotation matrix:

    vec3 ww = normalize(target - camera);
    vec3 uu = normalize(cross(ww, top);
    vec3 vv = normalize(cross(uu, ww));

where uu, vv, ww are rows or rotation matrix

[buddhi1980]

Example of this calculation of rotation matrix is used in my shader in ShaderToy: https://www.shadertoy.com/view/WlSSWW

[adrianmeyerart]

alright thanks. let me know

[buddhi1980]

In last commit I have added camera top vector to the image metadata

[adrianmeyerart]

thx, will try it out once the CI build is ready

[adrianmeyerart]

works! new tool completed and pull request send.

PS. exr/rotation metadata is incorrect and I don't use it anymore.

[buddhi1980]

Can I close this issue?

[adrianmeyerart]

yep!