search

ShenhanQian / VHAP

A complete head tracking pipeline from videos to NeRF/3DGS-ready datasets.
Other
95 stars 12 forks source link

Questions about cropping the video #8

Closed HaoZhiG closed 1 month ago

HaoZhiG commented 1 month ago

Thanks for release! I was wondering if you have considered cropping only the neck to head portion from the NeRSemble dataset, without including the shoulders?

ShenhanQian commented 1 month ago

It's possible to do so with predicted semantic masks. But these masks are not always robust and depend on the shape of the collars. Therefore, we fit a line, which is almost consistent across views and subjects, and mask out the region below.

Meanwhile, if we raise the line to a higher position, such as the neck, we may accidentally crop out the jaw in side views.

vipinsdk commented 1 month ago

Hello Shenhan,

Is it recommended to run the pipeline without cropping the videos to the shoulder region. I am asking this because the camera intrinsics are computed on the actual video size and if we cut the video then the intrinsics will also change right? If we were to hypothetically detect the shoulder region and fit a line, is this recommended?

Thanks in advance.

With regards, Vippin

ShenhanQian commented 1 month ago

Hi Vippin, our cropping only happens during export after tracking. You do not need to modify the video nor the camera intrinsics.