Homography extraction for some matches

[AhmedKhaled945]

Hello Dr.Chen, I reproduced the results found in the paper, by following the procedures you mentioned, but i found some matches that cannot be mapped correctly with our homography extraction module, They tend to have a different camera positioning i think, my hypothesis is that this types of camera positioning is not found in the database you provided, So my question is, is there a way that we can address this problem, as the database doesn't cover all possibilities, Thanks in advance.

[lood339]

Hi, Thanks for interesting in our work. Before I answer the question, I would like to know whether the line detection method works well or not work on your data. If the line detection method works well, we can enlarge the coverage by setting some parameters. There is function ut_generate_ptz_cameras in https://github.com/lood339/SCCvSD/blob/master/python/util/synthetic_util.py. The most important camera parameter is cc_mean which the camera center. If your camera is not near the middle line of the soccer field, you need to change this cc_mean to the location of your camera. The location is just a rough estimation of the camera, no accurate location is needed. BTW, it will be helpful if you can pose an example image so that I can know your problem better. Hope that helps.

[AhmedKhaled945]

Hello Dr Chen, i appreciate the answer, here is a frame example of a match that fails

The edge map extracted is

and the Retrieved image based on the matched features from the database is

As you can see, the zoom component is very different, the frame i provided is zooming in more than the one in the retrieved image, this zoom in is a stadium characteristic.

but still, the retrieved image covered the same area found in the frame, the only problem is it added to it, which gave bad mapping,

So how can we address such a problem? Thanks in advance, really appreciate the help.

[lood339]

Thanks for the images. They are very helpful. Here are two factors that fail the homography estimation. 1. The edge image is not good enough, for example, it has the goal gate and some irrelevant edges. It is because the training game (FIFA 2014) and the testing game is very different. 2. The retrieved edge image is very different from the real edge image. Two possible reasons, one is that the training set do not have so large zoomed image. I expect this is the main reason. Another reason is also from poor edge detection. For this image, one thing you can try is to increase the focal length variance in the training set. There is parameter fl_mean (see https://github.com/lood339/SCCvSD/blob/master/data/read_me.py line 21). Increasing this value in the sampling may generates the zoom-in views as your example image. The side effect of large zoom is that it may also generate 'empty' images, for example, zoom-in some no-line areas. For debug/test purpose, I would also suggest you (manually) clean the detected edge image first. Then, see if the query image becomes better. This is a way to test whether the main problem is from edge image quality or camera pose database. Hope that helps.

[AhmedKhaled945]

Thanks Dr Chen for the reply, it was really helpful, and i really think it is the edge detection, i will try to enhance this, any suggestions?

[lood339]

The bad detection is mainly because of the image difference between training and testing data. Data annotation can solve this problem. 1, annotate some edge images from your own match. It can be done by cleaning some edge images from the detection result. 2, add these cleaned edge image as training data and retrain the edge detection model. 3, test and see the new result.

For step 1, a better way is using a homography annotation tool. I had one before but it does not work well because of the dependence. May release a modified version later.