Using perspective projection

[trisct]

Hi, thanks for the great work.

I was looking through your code. It seems that projection_mode can be set in the models and the datasets, and is by default 'orthogonal'. However, it seems that projection_mode is only saved as attributes of the datasets and the models, and is not used in rendering (there is no projection_mode in apps/prt_util and apps/render_data).

So how did you deal with perspective projection, if the rendered images are all orthogonal projections? Do you transform the images when loading the dataset (sorry I haven't had enough time to look through the dataset code), or is this not implemented yet?

[trisct]

I also want to know if PIFu will work correctly if I simply set projection_mode to 'perspective', while leaving the rendered images orthogonal as it originally is.

[shunsukesaito]

The released code does not include data generation for perspective model. Unlike orthogonal, supporting perspective projection needs extra care although the inference code itself can trivially support perspective projection. My comment in the other issue below might be relevant. Also you can find other issues in both PIFu and PIFuHD regarding projection.

https://github.com/shunsukesaito/PIFu/issues/83

[ryancll]

I also want to know if PIFu will work correctly if I simply set projection_mode to 'perspective', while leaving the rendered images orthogonal as it originally is.

Hi @trisct , Did you figure out how to train the model with perspective mode? Do we need to reprocess camera parameters in TrainDataset?

[sinAshish]

following