LoFTR descriptors (feature representation retrival)

[DavidTu21]

Hi there,

Thank you first for all this amazing work!

I just had a quick question in regards to the feature representation that you used before and after the fine-matching step. In a normal pose estimation in the motion tracking workflow, we perform the 2D-2D intra-frame matching -> Triangulate matches to get 3D points -> Store those 3D points in the database, in the form of [x,y,z,1] and its descriptors (for example, 128 in length for SIFT) -> Match each stored data point with matches in the next frame to see if there are any matches.

I'm wondering is there a way that we could swap out SIFT matching pipeline and then use LoFTR? In order to achieve this, shall we consider using the LoFTR descriptors, if any?

[alien19]

Hi @DavidTu21 , Have u got any information?

Any help from LoFTR contributers will be appreciated! @JiamingSuen

[atharvjairath]

Hi @DavidTu21 , Have u got any information?

Any help from LoFTR contributers will be appreciated! @JiamingSuen

Same!

[TonyPhh]

Hi there,

Thank you first for all this amazing work!

I just had a quick question in regards to the feature representation that you used before and after the fine-matching step. In a normal pose estimation in the motion tracking workflow, we perform the 2D-2D intra-frame matching -> Triangulate matches to get 3D points -> Store those 3D points in the database, in the form of [x,y,z,1] and its descriptors (for example, 128 in length for SIFT) -> Match each stored data point with matches in the next frame to see if there are any matches.

I'm wondering is there a way that we could swap out SIFT matching pipeline and then use LoFTR? In order to achieve this, shall we consider using the LoFTR descriptors, if any?

hi，you can refer to this paper https://arxiv.org/abs/2207.03539

[FujiwaraZayako]

嘿，你好 首先感谢您所有这些出色的工作！ 我刚刚有一个关于您在微调步骤之前和之后使用的特征表示的快速问题。在运动跟踪工作流程中的正常姿势估计中，我们执行 2D-2D 帧内匹配 ->三角测量匹配以获得 3D 点 -> 以 [x，y，z，3] 及其描述符的形式将这些 1D 点存储在数据库中（例如，SIFT 的长度为 128）-> 将每个存储的数据点与下一帧中的匹配项进行匹配，以查看是否有任何匹配项。 我想知道有没有办法交换 SIFT 匹配管道，然后使用 LoFTR？为了实现这一点，我们是否应该考虑使用 LoFTR 描述符（如果有的话）？

嗨，您可以参考这篇论文 https://arxiv.org/abs/2207.03539

RWT-SLAM utilizes the keypoints and descriptors from the reformed LoFTR, and trained the vocabulary model by itself.Looks very successful, but not open source.Do you know of any other ways to use Loftr to slam?

[wappints]

Bump