google / hypernerf

Code for "HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields".
https://hypernerf.github.io
Apache License 2.0
895 stars 105 forks source link

Why are interpolated video_paths of HyperNeRF data not utilized? #51

Closed cmh1027 closed 9 months ago

cmh1027 commented 9 months ago
    def generate_video_path(self):
        self.select_video_cams = [item for i, item in enumerate(self.all_cam_params) if i % 1 == 0 ]
        self.video_path, self.video_time = smooth_camera_poses(self.select_video_cams,10)
        self.video_path = self.video_path[:500]
        self.video_time = self.video_time[:500]

    def load_video(self, idx):
        if idx in self.map.keys():
            return self.map[idx]
        camera = self.all_cam_params[idx]
        w = self.image_one.size[0]
        h = self.image_one.size[1]
        time = self.video_time[idx]
        R = camera.orientation.T
        T = - camera.position @ R
        FovY = focal2fov(camera.focal_length, self.h)
        FovX = focal2fov(camera.focal_length, self.w)
        image_path = "/".join(self.all_img[idx].split("/")[:-1])
        image_name = self.all_img[idx].split("/")[-1]
        caminfo = CameraInfo(uid=idx, R=R, T=T, FovY=FovY, FovX=FovX, image=self.image_one_torch,
                              image_path=image_path, image_name=image_name, width=w, height=h, time=time, mask=None
                              )
        self.map[idx] = caminfo
        return caminfo

These are functions in hyper_loader.py and it seems that only interpolated timestamps are used for video and interpolated poses are discarded. Could you explain me about the reason for it?

cmh1027 commented 9 months ago

Sorry. This issue was intended to be posted in GaussianSplatting4D repo.