XuyangBai / TransFusion

[PyTorch] Official implementation of CVPR2022 paper "TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers". https://arxiv.org/abs/2203.11496
Apache License 2.0
613 stars 76 forks source link

Initialisation of `gaussian_mask` in `TransFusionHead.forward_single` is not consistent with paper. #94

Open wqueree opened 1 year ago

wqueree commented 1 year ago

In the original TransFusion paper: it states that the Gaussian Mask, $M$, is initialised as follows, with inspiration from CenterNet:

$$ M_{ij}=\exp(-\frac{(i-c_x)^2 + (j-c_y)^2}{\sigma r^2}) $$

However, upon inspection of the code in transfusion_head.py, the following is found :

radius = torch.ceil(corners.norm(dim=-1, p=2) / 2).int()  # radius of the minimum circumscribed circle of the wireframe
sigma = (radius * 2 + 1) / 6.0
distance = (centers[:, None, :] - (img_feat_pos - 0.5)).norm(dim=-1) ** 2
gaussian_mask = (-distance / (2 * sigma[:, None] ** 2)).exp()

which is equivalent to:

$$ M_{ij}=\exp(-\frac{(i-c_x)^2 + (j-c_y)^2}{2\sigma^2}) $$

as per Objects as Points, which initialises $\sigma$ as per CornerNet.