Code for pre-filtering

[NehalD99]

Hi, I would like to train your model on using TildeP24 as standard patches on a different dataset. The paper mentions pre-filtering certain standard patches. It would help us if you kindly mention the pre-filtering steps and if possible provide a code for it.

Thank you.

[spongezhang]

Hi,

I don't remember where we mentioned pre-filtering in the paper. I think we just extract a 51*51 image patch around the Tilde point. You may also try multi-scale, which rescales the previous image by 1/sqrt(2). However, based on our experience, it doesn't help a lot.

Best, Xu

[NehalD99]

Hello Xu, Thank you for responding. As per your paper, on page 6, first paragraph it is mentioned to pre-filter candidate standard patches inorder to eliminate standard patches that lack cues to regress any required transformation. So I assume that you prefilter the tilde patches accordingly, and select some 5K patches for training. Also in your codebase, your network is set to regress translation parameters? I would like to know if it can be modified for other transformations. Kindly correct me if I have interpreted wrongly anywhere. Thank you, Nehal

[spongezhang]

Hi Nehal,

You meant that sentence. We just select top 5000 patches with the highest score. I think the training is stable with TILDE patches even with different images. It may have problem when using random patches or custom defined patches.

For other type of transform, I'm sorry that I've been working with different deadline during the past months. The code needs further polish. I'll get it ready by the end of next month. I'll keep this issue in the repository.

I don't think there has any difficulties in applying this to other transforms. One thing is that estimating translation and rotation separately may improve the performance.

Best,

[NehalD99]

Hello Xu,

I'm very grateful for the contribution of your codebase as it is. So I understand that practically estimating translation or rotation separately may be better in performance than estimating the combined transformation but theoretically if we are able to train on patch pairs with a general transform(eg.affine) between them, performance in combined case should also be good?

Thanks, Nehal

[spongezhang]

Theoretically, the parameters of the transformation should be trained together. However, the problem right now is the way we used to represent the transformation (as a 23 or 33 matrix). For example, both rotation and translation are all numbers in the matrix. However, their physical meanings are very different, which leads the magnitudes of the parameters are also very different. :(