search

szymanowiczs / splatter-image

Official implementation of `Splatter Image: Ultra-Fast Single-View 3D Reconstruction' CVPR 2024
https://szymanowiczs.github.io/splatter-image
BSD 3-Clause "New" or "Revised" License
795 stars 54 forks source link

Question about pose #16

Closed johnren-code closed 6 months ago

johnren-code commented 7 months ago

Thank you for your outstanding work!

You mentioned that when rendering images from other viewpoints, for camere2world, the camera pose uses the equivalent of the relative pose of the first frame of the image, if I want to render the first frame of the image at the same time, should I also use the relative pose of the first frame of the image directly? (i.e. unit matrix)

Also, did 3DGS originally work using the camera relative pose as well?

Looking forward to your reply!

szymanowiczs commented 6 months ago

The code already renders the image from the source camera during training here: https://github.com/szymanowiczs/splatter-image/blob/00fe5ef7874da71b178a531390aaafa5fa786a72/train_network.py#L208C13-L219C82 because the source cameras are actually duplicated in data, for example see here https://github.com/szymanowiczs/splatter-image/blob/00fe5ef7874da71b178a531390aaafa5fa786a72/scene/srn.py#L142.

Assuming data["view_to_world_transforms"][0] (the camera pose used to transform points from camera space to world space) is unit matrix then rendering from unit matrix will render source frame.

3DGS uses COLMAP poses which are relative by definition, as all camera registrations are equivalent to a rigid body transformation. So in short, yes, they are relative.