Question about data format

[VladimirYugay]

I want to render a replica dataset with the exact same format as used in iMap.

So far I was able to generate a smooth trajectory, navigate the agent along it, and store the respective depth maps and images which look nice. However, I'm failing when trying to run a method on it, while it works perfectly fine on the replica data from iMap.

I'm a little bit confused about how exactly to test/verify that my poses are legit. The snippet I used to get them assuming that there's a legit trajectory:

  for ix, point in enumerate(tqdm(path_points)):
      if ix >= len(path_points) - 1:
          break

      tangent = path_points[ix + 1] - point
      agent_state.position = point

      tangent_orientation_matrix = mn.Matrix4.look_at(point, point + tangent, np.array([0, 1.0, 0]))
      tangent_orientation_q = mn.Quaternion.from_matrix(tangent_orientation_matrix.rotation())
      agent_state.rotation = utils.quat_from_magnum(tangent_orientation_q)
      agent.set_state(agent_state)

      pose = np.eye(4)
      pose[:3, :3] = qt.as_rotation_matrix(agent_state.rotation)
      pose[:3, 3] = agent_state.position
      poses.append(pose)

[Harry-Zhi]

Hi @VladimirYugay , as far as I know the iMAP data from replica are also generated via randomly sampling $N$ camera positions and their corresponding $N$ look-at points, but with less unusual drastic motions compared to those in Semantic-NeRF to the sake of supporting camera tracking in SLAM . The complete trajectory with $M$ frames are obtrained by interpolation between these N keypoint poses.

I did not have a concrete idea whether the traj is legit or not. But if you want to run SLAM on it, I assume it would be good to produce a relative smooth trajectory like iMAP with a more constraint random generation process.