possible mistake in apply_camera_motion

[arnaud-nt2i]

Hi @viplix3

Your code in track.cpp/apply_camera_motion is different than the official Botsort https://github.com/NirAharon/BoT-SORT/blob/main/tracker/bot_sort.py#L68 and yolo_tracking repo https://github.com/mikel-brostrom/yolo_tracking/blob/master/boxmot/trackers/botsort/bot_sort.py#L96

your code: Eigen::Matrix<float, 8, 8> R8x8 = Eigen::Matrix<float, 8, 8>::Identity(); R8x8.block(0, 0, 2, 2) = R; mean = R8x8 mean.transpose(); mean.head(2) += t; covariance = R8x8 covariance * R8x8.transpose();

The other: R = H[:2, :2] R8x8 = np.kron(np.eye(4, dtype=float), R) t = H[:2, 2] for i, (mean, cov) in enumerate(zip(multi_mean, multi_covariance)): mean = R8x8.dot(mean) mean[:2] += t cov = R8x8.dot(cov).dot(R8x8.transpose()) stracks[i].mean = mean stracks[i].covariance = cov

So it seems you are applying camera motion compensation only to x,y ( and not w,h) while original code apply rotation and shear transform to x,y, w and h. See np.kron() for Kronecker product. Is it a mistake, or there is something else ?

Sorry for editing, code writing is broken in github

[arnaud-nt2i]

Correct code should be :

Eigen::Matrix<float, 8, 8> R8x8 = Eigen::Matrix<float, 8, 8>::Identity(); R8x8.block(0, 0, 2, 2) = R; R8x8.block(2, 2, 2, 2) = R; R8x8.block(4, 4, 2, 2) = R; R8x8.block(6, 6, 2, 2) = R; mean = R8x8 mean.transpose(); mean.head(2) += t; covariance = R8x8 covariance * R8x8.transpose();

[viplix3]

I will look into it this weekend