search

hustvl / 4DGaussians

[CVPR 2024] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
https://guanjunwu.github.io/4dgs/
Apache License 2.0
2.25k stars 186 forks source link

handling custom camera intrinsics #112

Closed hdacnw closed 7 months ago

hdacnw commented 8 months ago

Hi there, thanks for the great work!

I was wondering if it's possible to handle different camera intrinsics per frame e.g. different fx, cx? Specifically, can I modify getProjectionMatrix in graphics_utils.py by supplying custom P matrix to achieve this?

Thanks again.

guanjunwu commented 8 months ago

Yes, I think you can do it. btw, if you use colmap format, the camera intrinsics are in cameras.bin or cameras.txt, you can custom your camera intrinsics in the file. Isn't it?

hdacnw commented 8 months ago

Thanks for the quick reply! I'll try converting to the colmap format. The data I'm using involves long focal lengths e.g. 1000 and principle points that are on the edge instead of center due to cropping. I'm modifying P in getProjectionMatrix as 

    P[0, 0] = 2.0 * znear / (right - left)
    # P[1, 1] = 2.0 * znear / (top - bottom)
    P[1, 1] = 2 * fy / h
    # P[0, 2] = (right + left) / (right - left)
    P[0, 2] = -(w - 2 * cx) / w
    # P[1, 2] = (top + bottom) / (top - bottom)
    P[1, 2] = -(h - 2 * cy) / h
    P[3, 2] = z_sign
    P[2, 2] = z_sign * zfar / (zfar - znear)
    P[2, 3] = -(zfar * znear) / (zfar - znear)

but model won't converge. Do you have any suggestions on what else I should try for it to work e.g. changing aabb bounds? Thanks!

Tonychen050400 commented 1 month ago

Hi, I face the similar question, do you solve this problem?