google-research / kubric

A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.
Apache License 2.0
2.35k stars 233 forks source link

Fix half-pixel error in query points for point tracking #284

Closed cdoersch closed 1 year ago

cdoersch commented 1 year ago

Fixes #280 : The underlying cause was that query points were output in pixel coordinates, rather than raster coordinates. Furthermore, for background points, the mapping between the query point and the depth passed a pixel coordinate into the inverse camera matrix, rather than a raster coordinate, meaning that the point being tracked was not necessarily even on a surface.

As a result, for objects, the query point is off by a half pixel, and for background points, the actual tracked point may be slightly off from the surface. This PR should fix both issues.