search

eriksandstroem / Point-SLAM

Point-SLAM: Dense Neural Point Cloud-based SLAM
Apache License 2.0
359 stars 29 forks source link

Meaning of saving r_query_idx.pt in dynamic_r_frame folder? #23

Closed mtk380 closed 5 months ago

mtk380 commented 6 months ago

I think dynamic_r_frame is related to the sampled rays and points along the rays, so the saved pt can not be used if we test the model. Am I understanding this correctly？

eriksandstroem commented 5 months ago

Hi @mtk380, Very sorry for the late reply here. I'm afraid I cannot understand what you mean from your question. Can you be more explicit and concrete with your question?

mtk380 commented 5 months ago

Thanks for your reply. 1、 In Mapper:line686 the code calculates and save dynamic_radius : if self.use_dynamic_radius: ratio = self.radius_query_ratio intensity = rgb2gray(gt_color.cpu().numpy()) grad_y = filters.sobel_h(intensity) grad_x = filters.sobel_v(intensity) color_grad_mag = np.sqrt(grad_x**2 + grad_y**2) # range 0~1 color_grad_mag = np.clip( color_grad_mag, 0.0, self.color_grad_threshold) # range 0~1 fn_map_r_add = interp1d([0, 0.01, self.color_grad_threshold], [ self.radius_add_max, self.radius_add_max, self.radius_add_min]) fn_map_r_query = interp1d([0, 0.01, self.color_grad_threshold], [ ratio*self.radius_add_max, ratio*self.radius_add_max, ratio*self.radius_add_min]) dynamic_r_add = fn_map_r_add(color_grad_mag) dynamic_r_query = fn_map_r_query(color_grad_mag) self.dynamic_r_add, self.dynamic_r_query = torch.from_numpy(dynamic_r_add).to( self.device), torch.from_numpy(dynamic_r_query).to(self.device) if init: torch.save( self.dynamic_r_query, f'{self.output}/dynamic_r_frame/r_query_{idx:05d}.pt') but the result should be same in different idx. 2、the dynamic_r_query is used in Mapper:844 , vis_value_only.
cur_frame_depth, cur_frame_color = self.visualizer.vis_value_only(idx, 0, gt_depth, gt_color, cur_c2w, self.npc, self.decoders, self.npc_geo_feats, self.npc_col_feats, freq_override=True, dynamic_r_query=r_query_frame, cloud_pos=self.cloud_pos_tensor, exposure_feat=self.exposure_feat_all[ render_idx // cfg["mapping"]["every_frame"] ].to(self.device) if self.encode_exposure else None) I donot find other code using and changing dynamic_r_query, so dynamic_r_query is all the same, why should we generate dynamic_r_query for every frame?

eriksandstroem commented 5 months ago

Thanks for your quick reply.

The dynamic search radius is actually not only frame specific but even pixel specific since it depends on the color gradient magnitude at the pixel. Therefore, the value is different for different indices.

Does this help?