Question about leveraging 3D information for the Query Image

WangYixuan12 / d3fields

[CoRL 24] D^3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Robotic Manipulation

https://robopil.github.io/d3fields/

MIT License

108 stars 6 forks source link

Question about leveraging 3D information for the Query Image #4

Closed AlbertoRemus closed 6 months ago

AlbertoRemus commented 9 months ago

Hi!

I have a question about the query image. Since it's RGB only, how can the 3D part of the features be leveraged to find 2D-3D correspondences? (in the block c of the figure)

274870606-f4aa138f-3f45-40a1-ac6e-2b521594da9a

WangYixuan12 commented 9 months ago

We construct a reference camera here that can map 3D keypoints to 2D space to compute MSE Loss