zhuhao-nju / facescape

FaceScape (PAMI2023 & CVPR2020)
833 stars 94 forks source link

Questions about Multi-view data camera params #117

Open ATPBoost opened 7 months ago

ATPBoost commented 7 months ago

How should I load Multi-view data camera params(params.json) for gaussian-splatting?

I loaded params.json using the following code, and I preprocess the image through cv2.undistort(image, intr, dist):

def readCamerasFromParams(path, white_background, extension=".png"):
    cam_infos = []
    uid = 0
    with open(os.path.join(path, "params.json"), 'r') as json_file:
        params = json.load(json_file) # read parameters
    for filename in os.listdir(path):
        if filename.endswith(extension):
            image_name = os.path.splitext(filename)[0]
            valid = params['%s_valid' % image_name]
            if valid:
                uid = uid + 1 
                intr = np.array(params['%s_K' % image_name]) # intrinsic mat
                w2c = np.array(params['%s_Rt' % image_name]) # extrinsic mat
                dist = np.array(params['%s_distortion' % image_name]) # distortion parameter
                height = params['%s_height' % image_name] # height
                width = params['%s_width' % image_name] # width
                R = np.transpose(w2c[:3, :3])
                T = w2c[:3, 3]

                FovX = focal2fov(intr[0, 0], width)
                FovY = focal2fov(intr[1, 1], height)

                # ......loading image......

                cam_info = CameraInfo(uid=uid, R=R, T=T, FovY=FovY, FovX=FovX, image=image, image_path=image_path, image_name=image_name, width=width, height=height)
                cam_infos.append(cam_info)

    return cam_infos

After training, the rendering results showed multi face overlap like this 0018 00032

How can I solve this problem?

Ling27 commented 7 months ago

@zhuhao-nju @ATPBoost I have the same question regarding this issue. Watching...