Potential Error in main_importKnownPoses.cpp

[mpr-projects]

Hello,

is it possible that there's a mistake in the file main_importKnownPoses.cpp?

I've imported my camera poses with the ImportKnownPoses node. There doesn't seem to be any documentation on the file format but from looking at the code in main_importKnownPoses.cpp it seems that the data should be in a .json file. The file should contain one line per image and each line is a json dictionary with three entries: 'pose', 'forward' and 'up'.

When I put my camera poses into that format (and after accounting for Meshroom's rotated coordinate system) then they import fine and I can see the camera poses in the 3D-viewer by double clicking on the node. But the cameras point in the wrong direction. If I flip the direction of the 'forward' vector and then the cameras point in the right direction but the two (red and green) lines indicating the local camera coordinate system point in the wrong directions (when compared to the result I get from a reconstruction without known poses). I've also tried swapping and flipping the signs of the inputs in as many ways as I could think of and it never seemed right (the reconstruction always failed or gave wrong results) -- of course I may have missed some combination of inputs ...

I'm not familiar with Meshroom's entire codebase so I'm not sure how exactly the coordinates are used/processed internally but I have the impression that line 372 of src/software/convertmain_importKnownPoses.cpp might have to be

rot.row(0) = -up_vec.cross(forward_vec);

instead of

rot.row(0) = up_vec.cross(forward_vec);

Otherwise we'd get a left-handed coordinate system (but I think Meshroom is using a right-handed one, right?). When I adjust the code with that change and compile and run it then the poses import correctly and the reconstruction works fine.

I hope that was detailed enough, otherwise, I describe it in more visually in a video I've just made about some aspects of using Meshroom (see https://youtu.be/XUKu1apUuVE). The section about using known poses starts at time 19:20, the ImportKnownPoses-related part at 22:20 (feedback is welcome :).

[natowi]

@fabiencastan

@mpr-projects great video!