Closed zjcs closed 6 months ago
By the way, it is a solution to #2
Thanks for your contribution!
@buaacyw sorry to say that the above code has some bug when calucate fov via aspect, here is the right implemention:
@self.server.on_scene_click
def _(point):
for client in self.server.get_clients().values():
camera = client.camera
ray_d = torch.Tensor(tf.SO3(camera.wxyz).inverse().as_matrix() @ point.ray_direction)
# ray_d, the ray in the camera, x->right, y->down, z->far
xy_norm = ray_d[:2]/ray_d[2]
WH = torch.Tensor([self.resolution_slider.value, self.resolution_slider.value/camera.aspect]).int()
tan_fov_2 = torch.Tensor([np.tan(camera.fov/2)*camera.aspect, np.tan(camera.fov/2)])
fxy = WH/2 / tan_fov_2
uv = (fxy * xy_norm + WH/2).int()
click_pos = uv / WH
@zjcs I modify the webui.py code as you provided, but the bounded box and SAM points still cannot be added into scene, what else should I need to change to let those functions work?
I notice that author modify the origin viser to capture the scene click pixel (actually, it is normal pixel). Here is an implemention via the origin viser, which can be used in click_cb function: