search

system123 / SOMatch

A Framework for Deep Learning-based Sparse SAR-Optical Image Matching
MIT License
48 stars 17 forks source link

about goodness net #9

Closed willa2016 closed 1 year ago

willa2016 commented 3 years ago

Sorry to bother you. I am confused about the goodness network since that the out put of the vggbasedgoodnessnet is a single number but how does the nms operation works then? do I have to close the F.adaptive-avg-pool2d option to use the fts tensor after fc2?

LiBudde commented 1 year ago

@willa2016 I have the same problem. For me it is also not clear how I get the cross-modality scene goodness and how I process further with the correspondance network after training the single networks seperatly. Do you find a solution for this?

system123 commented 1 year ago

During inference pass the parameter "pool=False" to the forward method of the "goodnessNet"

net = VGGBasedGoodnessNet(...)
goodness_heatmap = net(x_patch, pool=False)

Once you have the SAR and Optical goodness heatmaps you merge them using a pixel-wise merge operator (such as min pixel, or max pixel) or using the fuse_goodness operation inside the VGGBasedGoodnessNet class object. Finally use extract_good_points operation to apply NMS and get a point list of the determined good points for matching.

All of this is explained in the paper.