"weight_sum" and "alpha_sum" become zero in generic process

[CCOSerika]

Hi, author. I run the generic and finetune code successfully on the DTU dataset. When it comes to customized dataset, the parameter "weight_sum" and "alpha_sum" become zero in generic process. So how to avoid this? By introducing "visibility_beta" and "visibility_weight_thred" or other way?

[CCOSerika]

Hi, author. I've solved the issue above and the extracted mesh become no more confusing during training. But the mesh I get seems to be incorrect and I meet "RecursionError: maximum recursion depth exceeded" during iter 39500. I guess it is caused by incorrect camera pose info or bad param setting. I want to ask how to get the "depth_min" and "depth_interval" in cam.txt, and which params need to be modified on different datasets?

[CCOSerika]

BTW, I noticed that the train and test datasets have different format. So how to create test dataset from train dataset. I use following code but failed.

def cal_scale_mat(img_hw, intrins, extrins, near_fars, factor=1.0):
    center, radius, _ = get_boundingbox(img_hw, intrins, extrins, near_fars)
    radius *= factor
    scale_mat = np.diag([radius, radius, radius, 1.])
    scale_mat[:3, 3] = center.cpu().numpy()
    return scale_mat.astype(np.float32)

def read_cam_txt(path):
    with open(path, 'r') as fr:
        lines = [line.strip() for line in fr.readlines()]

    extrin = np.fromstring(' '.join(lines[1:5]), 
        dtype=np.float32, sep=' ').reshape(4, 4)

    intrin_t = np.fromstring(' '.join(lines[7:10]),
        dtype=np.float32, sep=' ').reshape(3, 3)
    intrin = np.eye(4, dtype=np.float32)
    intrin[:3, :3] = intrin_t

    depth_min = float(lines[11].split(' ')[0])
    depth_interval = float(lines[11].split(' ')[1])
    depth_max = depth_min + depth_interval * 192.
    return extrin, intrin, [depth_min, depth_max]

def write_cam_npz(path):
    extrins, intrins, near_fars = [], [], []
    for idx in [22, 24, 25]:
        meta = read_cam_txt(osp.join(path, f"{idx:08d}_cam.txt"))
        extrins.append(meta[0])
        intrins.append(meta[1])
        near_fars.append(meta[2])

    scale_mat = cal_scale_mat([512, 640], intrins, extrins, near_fars, factor=1.1)

    world_mats = []
    for intrin, extrin in zip(intrins, extrins):
        world_mats.append(intrin @ extrin @ scale_mat)

    np.savez(osp.join(path, "cameras.npz"), 
        world_mat_0=world_mats[0], 
        world_mat_1=world_mats[1], 
        world_mat_2=world_mats[2])

[Liangsanzhu]

Hello! How do you solve the problem of "weight_stum" and "alpha_stum"? This is very important to me and I am looking forward to your reply.

Hi, author. I've solved the issue above and the extracted mesh become no more confusing during training. But the mesh I get seems to be incorrect and I meet "RecursionError: maximum recursion depth exceeded" during iter 39500. I guess it is caused by incorrect camera pose info or bad param setting. I want to ask how to get the "depth_min" and "depth_interval" in cam.txt, and which params need to be modified on different datasets?