search

shunsukesaito / PIFu

This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"
https://shunsukesaito.github.io/PIFu/
Other
1.76k stars 341 forks source link

Multi-views evaluation #31

Closed waldstein94 closed 4 years ago

waldstein94 commented 4 years ago

Hello, I have questions about the way of evaluation with multi-views inputs.

After reading your paper, I couldn't understand the procedure of evaluation with multi-view inputs. For training phase with multi-views input, we know corresponding pixel (xi, noted in your paper) between images from different view point for a single point X on the 3d model. However, for test phase, we don't know any 3d information or the relation between images. So, how do I pick the set of xi for a single voxel in 3d occupancy field in test phase?

I'm sorry to bother you if the answer could have been obtained from your paper.

shunsukesaito commented 4 years ago

PIFu assumes you have camera calibration between multiple images for both training and test time. So during test time, you just need to query 3D points X within predefined 3D bounding box, then using the camera matrices, you should be able to obtain pixel coordinate x as well as depth for X.

waldstein94 commented 4 years ago

Oh, you mean we have to know the rotation degree of each image to obtain the camera matrices? Because I want to apply your code with multi-views images from video data, I should know detailed process in evaluation. Also, I think the function of gen_mesh is only for single-view input. Is it right?

shunsukesaito commented 4 years ago

Oh, you mean we have to know the rotation degree of each image to obtain the camera matrices?

In our camera setup, indeed only rotation angles are the changing variables. But generally speaking, you need to know scale, rotation matrix, and translation for weak perspective model.

Also, I think the function of gen_mesh is only for single-view input. Is it right?

Correct. You need to modify the code to incorporate multiple images.

alitokur commented 4 years ago

@waldstein94 hi, did you modify the gen_mesh function?