search

flex-3dat / 4DGaussians

https://guanjunwu.github.io/4dgs/
Other
0 stars 0 forks source link

The poor rendering quality from Non-Dataset viewpoints #4

Open imjxy opened 6 months ago

imjxy commented 6 months ago

Use a Better Dataset ✔️

Using a better dataset can effectively improve the quality of each generated 3D Gaussian at every timestamp. A new dataset was captured in a spacious and well-lit area, ensuring that the human body is completely in the frame at all times. We are planning to capture another dataset that ensures the same conditions while also including more camera angles and the human body rotating. Previous Dataset (in a Stairwell): image New Dataset: image

imjxy commented 6 months ago

Clear Background Information❓

Clearing background information from the current dataset did not result in a noticeable improvement in reconstruction quality. This might be related to the method used for background removal; currently, the REMBG Python library is used, but arms are often misidentified as background and removed. Additionally, the images with cleared backgrounds are difficult to use with COLMAP for calculating camera intrinsics and extrinsics. Therefore, the camera intrinsics and extrinsics were calculated using the images before background removal, and the training was performed using the images after background removal.

Dataset with Background Information: image

Dataset without Background Information: image

imjxy commented 6 months ago

Clear Background Information❓

Clearing background information from the current dataset did not result in a noticeable improvement in reconstruction quality. This might be related to the method used for background removal; currently, the REMBG Python library is used, but arms are often misidentified as background and removed. Additionally, the images with cleared backgrounds are difficult to use with COLMAP for calculating camera intrinsics and extrinsics. Therefore, the camera intrinsics and extrinsics were calculated using the images before background removal, and the training was performed using the images after background removal.

Dataset with Background Information: image

Dataset without Background Information: image

When I used a more advanced segmentation network (DeepLabV3), the reconstructed 3D Gaussians did not perform better, but the rendered output video did show clearer arm movements.