How can I compute the intrinsics of G-objaverse?

[thucz]

There are x_fov and y_fov in the camera parameters json file. I want to get the intrinsics and camera2world parameters. I don't know how to get f_x and f_y. Are x_fov and y_fov in Radian format? fx = 0.5 * W * 1 / np.tan(0.5 * xfov) or fx = xfov * W? Do you know which way is correct?

The code I plan to use:

    with open(json_file, 'r', encoding='utf8') as reader:
        json_content = json.load(reader)
    # compute intrinsics
    assert len(img_size) == 2
    H, W = *img_size 
    H, W = 
    cx = (W - 1) / 2.0
    cy = (H - 1) / 2.0
    xfov, yfov = json_content['x_fov'], json_content['y_fov']
    # fx, fy = xfov * W, yfov * H
    fx = 0.5 * W * 1 / np.tan(0.5 * xfov)
    fy = 0.5 * H * 1 / np.tan(0.5 * yfov) 

    K = np.array([
            [fx, 0, cx],
            [0, fy, cy],
            [0, 0,  1],
        ], np.float32)

    return camera_matrix, K #intrinsics

[lingtengqiu]

Given json file in folder, for example, 00000.json The following code can obtain camera to world matrix

    camera_matrix[:3, 0] = np.array(json_content['x'])
    camera_matrix[:3, 1] = np.array(json_content['y'])
    camera_matrix[:3, 2] = np.array(json_content['z'])
    camera_matrix[:3, 3] = np.array(json_content['origin'])

[thucz]

Given json file in folder, for example, 00000.json The following code can obtain camera to world matrix

    camera_matrix[:3, 0] = np.array(json_content['x'])
    camera_matrix[:3, 1] = np.array(json_content['y'])
    camera_matrix[:3, 2] = np.array(json_content['z'])
    camera_matrix[:3, 3] = np.array(json_content['origin'])

Thanks for your reply! But I would like to ask how to get the intrinsics?

[thucz]

I want to warp view1 to view2 with the given depth(I used original depth instead of disparity for simplicity), poses and rgbs. But I got the wrong projection results:

view1:

view2:

The warp img from view1 to view2:

The code I used: https://gist.github.com/thucz/20f2dffe6010154344bfd7c04e2eb085

I run python pose_debug.py, and aux.py is an auxiliary file for visualization and warping, and I put aux.py under the same directory with pose_debug.py.

data_root = "./gobjaverse/0/10010/" is the data I used.

Do you know where the problem is in my projection code?

[lingtengqiu]

The instrinsic can be defined by the following code:

def get_intri(target_im):
        h, w = target_im.shape[:2]

        fx = fy = 1422.222
        res_raw = 1024
        f_x = f_y = fx * h / res_raw
        K = torch.tensor([f_x, 0, w / 2, 0, f_y, h / 2, 0, 0, 1]).reshape(3, 3)
        # print("intr: ", K)
        return K

I test an example: given a ref view depth and rgb, this demo warps ref view to target view. Note using single-edge depth warpping, there will be some noises when the target views are wide apart.

For example:

where, from left to right, target view, warpping img, ref img, merge warpping img into target view, visualization from ref to target.

Please see https://github.com/modelscope/richdreamer/tree/main/dataset/gobjaverse, Depth-Warp Session

[thucz]

Thanks!