Generate the rendered image from perspective point of view

[etudemin]

Hi,

Firstly, thanks for the amazing library!

I want to extract the rendered image from perspective point of view given the mesh (already transformed from the world coordinate to the camera coordinate) and camera intrinsic parameters (fx, fy, cx, cy). In addition, it would be better to generate the depth map together.

After digging into the source code, I find that

render(core::Mesh mesh, glm::tmat4x4<float> model_view_matrix, glm::tmat4x4<float> projection_matrix,
       int viewport_width, int viewport_height, const cpp17::optional<Texture>& texture = cpp17::nullopt,
       bool enable_backface_culling = false, bool enable_near_clipping = true,
       bool enable_far_clipping = true)

in render.hpp or

eos::core::Image4u extract_texture(const core::Mesh& mesh, glm::mat4x4 view_model_matrix, glm::mat4x4 projection_matrix,
                        glm::vec4 /*viewport, not needed at the moment */, const eos::core::Image4u& image,
                        bool /* compute_view_angle, unused atm */, int isomap_resolution = 512)

in texture_extraction.hpp might be the solution. But I am not so sure about the completion of these two functions.

Excuse me, is there any suggestions of how to achieve the perspective rendering and perspective depth map generation?

Looking forward to your reply, thank you very much.

[patrikhuber]

Hi,

Thanks for the very nice words.

eos estimates an orthographic camera, which you can't "convert" to a perspective camera. However you can take an estimated mesh, and render it with any model-view and projection matrix. The projection matrix can be orthographic or perspective - the renderer in eos should follow the OpenGL convention pretty closely. You can for example construct a matrix with glm::perspective(...) and pass that to eos::render::render(...) as projection_matrix. You can also have a look at these lines in patrikhuber/4dface - the code is a bit dated and uses an older version of eos (please don't use that!), but you can see there how to render a fitted mesh in a different pose.

Perhaps you may also want to look into fit-model-ceres and write your own cost function, given your camera setup/parameters? You could take the existing code as an example.

Hope that helps.

[etudemin]

Hi,

Thank you very much, that really helps. :)

Sincerely,