Closed guihaik closed 7 months ago
@konglinhua 抱歉,目前还真没做过这块的可视化。我们可视化一般看的是decoder中每个object query投影过来的attention可视化。我想是不是可视化之前需要对可视化做个norm,有些数值比较大,导致可视化效果不好?
嗯嗯,这个是对图像和Heatmap归一化之后按照权重相加的结果,我还尝试了下用torch中提供的R50对这个图像可视化看下,结果看着合理一些,就一直很纳闷。还想请教一下,你们可视化的attention是针对下边函数中的weight吗?
def forward(self, instance_feature, query_pos,feat_flatten, reference_points, spatial_flatten, level_start_index, pc_range, lidar2img_mat, img_metas):
bs, num_anchor = reference_points.shape[:2]
reference_points = get_global_pos(reference_points, pc_range)
key_points = reference_points.unsqueeze(-2) + self.learnable_fc(instance_feature).reshape(bs, num_anchor, -1, 3)
weights = self._get_weights(instance_feature, query_pos, lidar2img_mat)
features = self.feature_sampling(feat_flatten, spatial_flatten, level_start_index, key_points, weights, lidar2img_mat, img_metas)
output = self.output_proj(features)
output = self.drop(output) + instance_feature
return output
‘’‘
@konglinhua deformable attention版本用的是这个weights。
请问有尝试过对backbone或者FPN的特征可视化吗,我尝试用EigenCAM可视化FPN后效果很差,但是NDS是54.2,用的config是repdetr3d_r50_704_bs2_seq_90e.py,把vov变成了r50
![0_1](https://github.com/exiawsh/StreamPETR/assets/33696039/9c9afdd0-93ca-4ddb-8922-2d1391dcf4cf)