search

placeforyiming / CVPR21-Deep-Lucas-Kanade-Homography

A generic pipeline to align multimodal image pairs from different sensors by extending Lucas-Kanade on feature maps. CVPR2021
128 stars 29 forks source link

How can I get the warped image with this code like demo.gif? #3

Closed qjr1997 closed 3 years ago

qjr1997 commented 3 years ago

I tried the “model_ours” to train “GoogleMap” . After that , I opened “Visualization.ipynb” and run it . I got the results like this. ec624ca062803c9e3253be1881d8e4d I want to ask that how can I get the true warped image with this code.

placeforyiming commented 3 years ago

I tried the “model_ours” to train “GoogleMap” . After that , I opened “Visualization.ipynb” and run it . I got the results like this. ec624ca062803c9e3253be1881d8e4d I want to ask that how can I get the true warped image with this code.

Hi,

The feature map has been extracted, then you need an initial guess of the homography and update the homography vector by using Lucas-Kanade on the feature map.

The initial guess can be:

  1. A reasonable guess, for example, assume two images share the same center.
  2. Classic methods, we tried the Fast-Match from the paper "Fast-match: Fast affine template matching" in the rebuttal.
  3. In this paper, we trained two different networks to provide the initial guess.
  4. If you want, you also can try if the traditional feature-based method, like harris corner, can provide a good initial

The better the initial guess, the larger chance Lucas-Kanade can converge.

qjr1997 commented 3 years ago

Thank you for your reply ， I still have some questions . “we trained two different networks to provide the initial guess” ，is that mean "DHN" and "MHN" mentioned in the paper ? Are they included in this code or I should find them from other places ？By the way I want to know what‘s the meaning of the parameter “regular” and “special” .

placeforyiming commented 3 years ago

Thank you for your reply ， I still have some questions . “we trained two different networks to provide the initial guess” ，is that mean "DHN" and "MHN" mentioned in the paper ? Are they included in this code or I should find them from other places ？By the way I want to know what‘s the meaning of the parameter “regular” and “special” .

You can find them in this code under the folder model_ours.

One of our contribution is the feature constructor, the special means the feature map extract by our feature constructor, the regular means the feature map is just the output of a regular one channel conv