Homography

[xiahaa]

@xiaohanlyu Homography and other two view geometry may be a little bit complex for you. So it is a good idea that you read the lecture note of UiO in advanced.

[xiahaa]

assignment

When you finished, tell me. Then we will combine feature detection, matching, RANSAC and DLT.

[hannaLyu]

I already finished warp.

[xiahaa]

I already finished warp.

ok. Then you need to implement a RANSAC framework for homography estimation. You have all the information you need on the slides I gave you. So you can start right now. However, if you are not clear about what to do exactly. I will provide you some information perhaps this night.

After this, tell me again your progress.

[hannaLyu]

Ok i will try to do it now.

[xiahaa]

Ok i will try to do it now.

for better results, you need to add the following issues:

1. normalization;
2. ransac;

see updated docs.

Do normalization and RANSAC.

When you finish, tell me.

You are approximating the finishing line. Bon courage.

[xiahaa]

see results:

[xiahaa]

Once you have done the normalization and ransac homography estimation. You should combine all things together.

1. take two images;
2. extract harris or fast, extract brief features;
3. match;
4. use ransac homography estimation;
5. warping; You can start with two images. Multiple images can be left as an extra assignment for next week.

[hannaLyu]

Actually it's still unclear for me to combine RANSAC and Hest. function ||x2−Hx1||+||x1−H^(-1)x2|| is that a function to calculate for what distance? In my mind, When i choose 4 points from matching pairs, I should calculate distance as before i do. or i just use this function?

[xiahaa]

Actually it's still unclear for me to combine RANSAC and Hest. function ||x2−Hx1||+||x1−H^(-1)x2|| is that a function to calculate for what distance? In my mind, When i choose 4 points from matching pairs, I should calculate distance as before i do. or i just use this function?

so for ransac, what you need?

1. minimum set of data (4 points here);
2. you need a model and you know how to fit (H and DLT);
3. metric that you can use to justify one point as inlier or outlier (for line fitting, you use point2line distance, for plane, you use point2plane distance, for homography, use ||x2−Hx1||+||x1−H^(-1)x2||). You can use a metric like ||x2−Hx1||, that should work also. However, in some famous books, ||x2−Hx1||+||x1−H^(-1)x2|| is the recommended metric.

[hannaLyu]

In my mind, i should use every ransac for each H matrix, like if i choose 500 times random correspondences and then get 500 numbers of H matrix, i will use RANSAC function for 500 times, But in this way my computer always calculate for a long time, so that's why i'm confusing. Maybe i should just choose only one H matrix and then go to ransac?

[xiahaa]

In my mind, i should use every ransac for each H matrix, like if i choose 500 times random correspondences and then get 500 numbers of H matrix, i will use RANSAC function for 500 times, But in this way my computer always calculate for a long time, so that's why i'm confusing. Maybe i should just choose only one H matrix and then go to ransac?

ransac

see my answer.

1. you should adjust the maximum iteration number automatically. usually it will not run more than 500 times.
2. I don't fully understand your post. So each time, you need to randomly sample 4 points, and then estimate a H using DLT, then use this H to all data to compute a metric in order to verify which is inlier and which is outlier. The three steps will iterate for a certain number of times.
3. Even if you use fixed 500 times, I wouldn't say it will cost you a long time (I would say less than 5 seconds). So maybe you can upload your code. Then we will say why it is so slow.

[hannaLyu]

I upload the unfinished code. I think the problem is still in distance. Cause when i use H to all data, there are a loop in original loop , so maybe i have a misunderstanding in this metric.

[xiahaa]

I upload the unfinished code. I think the problem is still in distance. Cause when i use H to all data, there are a loop in original loop , so maybe i have a misunderstanding in this metric.

 for i=1:iter ????? why i:iter????
   a=H* X1(:,i);
   b=inv(H)*X2(:,i);
   distance(:,i)=norm(X2(:,i)-a)+norm(X1(:,i)-b);
  end

shouldn't it be

for i = 1:number1
    a = H*X1(:,i);
    b = inv(H)*X2(:,i);
    distance(:,i)=norm(X2(:,i)-a)+norm(X1(:,i)-b);
end

no need for loop since you are using homogeneous cooridnates

X1t = H*X1;
X2t = H\X2;
X1t = X1t./X1t(3,:);
X2t = X2t./X2t(3,:);% make 3rd coordinate being one. Why? Think about this.
distance = sqrt(diag((X1t - X1)'*(X1t - X1)))+ sqrt(diag((X2t - X2)'*(X2t - X2))); % why this? think about this.

[hannaLyu]

I still have question in distance, i plan to ask you tomorrow afternoon cause i have a class in the morning,

[xiahaa]

I still have question in distance, i plan to ask you tomorrow afternoon cause i have a class in the morning,

ok.

[xiahaa]

@xiaohanlyu 这个老师Christensen, David Johan以前有门课程叫什么modular robotics什么的，好像还挺有意思的。不过他离职去创业了。

[xiahaa]

To improve your interest, I suggest take some images using your phone, e.g. DTU 101 if you like?, and then make your own panorama. The tips for taking images:

1. enough overlaps, at least 30% between frames;

[hannaLyu]

I create a new folder for my stitching codes. It seems inliers is ok but there has an error in Warp function. Maybe that's because my H matrix? I can't find bug now.

[xiahaa]

1. see the first commit things changed: 1. use brief 256; 2. only consider match is a good match if the best matched score is 0.7 times smaller than the second best matched score. 3. You have a bug, you only check for u1 v1 but not for u2 v2

[xiahaa]

2. see the second commit. You have made a siginifcant bug on RANSAC for homography.

   idx1 = randperm(number1,mini);
   x1 = X1(:,idx1);
   [T1,x1h] = normalization(x1);

   Why add this idx2 = randperm(number2,mini);x2 = X2(:,idx2); ? X1 X2已经成对了，你随机采样了x1，那么对应的x2就出来了。 你对二者都随机采样，不就打破了对照关系了么，那还能估计正确的H么？

第二个bug distance = sqrt(diag((X1t - X1)'*(X1t - X1)))+ sqrt(diag((X2t - X2)'*(X2t - X2))); X1t = HX1 那么X1t是和X2对应，你又把X1t和X1做差，这是什么意思？

[xiahaa]

3. see the third commit. 需要将前两行互换，因为feature存的是v u，而warp的时候是按照u v来变换的。

[xiahaa]

It seems that you still have some problems with understanding RANSAC~

[hannaLyu]

现在我对ransac的理解：随机取一对比较对的坐标x1,x2，把x1进行homograph变换， 将变换后的坐标x1'与x2做差，寻找可以使最多变换后的坐标x1'接近原比较对中的x2的H矩阵. 这样对吗？

[hannaLyu]

我拍了图书馆的照片，但是运行速度太慢，就没继续下去

[xiahaa]

现在我对ransac的理解：随机取一对比较对的坐标x1,x2，把x1进行homograph变换， 将变换后的坐标x1'与x2做差，寻找可以使最多变换后的坐标x1'接近原比较对中的x2的H矩阵. 这样对吗？

1. 针对这个问题，你这么理解是对的。 这里x1 x2是成对支持H的，式子是x2^T-Hx1 = 0，你可以抽象的写作f(H,x')，这里x'=(x1,x2)，x'是x1 +对应于x1的x2。

比如你哪天遇到一个问题，是x3 + Hx1 - x2 = 0，那么x'是x1+x1对应的x2+x1对应的x3。

2. 将变换后的坐标x1'与x2做差，寻找可以使最多变换后的坐标x1'接近原比较对中的x2的H矩阵. 这样对吗？

要掌握原理，不是死记这个式子。还是这个例子x3 + Hx1 - x2 = 0，那么假如你算了一个H，怎么再去计算一个metric判断：对于当前的H，某一个x'={x1 x2 x3}是不是inlier? 你要把metric设计成x3 + Hx1 - x2。

[xiahaa]

我拍了图书馆的照片，但是运行速度太慢，就没继续下去

imresize，downsize一下你的图像。 你手机拍的应该在2000万像素，太大了。

[xiahaa]

这个poisson blending可以做无缝的图像融合。 好像ps里面的智能剪刀后面就是这个算法。 跟课程没啥关系，你感兴趣就自己找几个图片试试。

[xiahaa]

另外，参照code。 尝试一下多图stitch。注意把相关代码换成你的code。