Is there a way to see the generated displacement field?

[Kunz-David]

Hello, I am working on an image registration method and Augmentor would be a nice way for me to test it. Ideally, however, I would like to see the displacement field that maps the original to the augmented image. Is there a way to do this in Augmentor?

For example is there a way to see a displacement vector for every pixel in img that map to the pixels' location in img_proccessed for every a specific augment call?

img = testpattern()
pl = FlipX() |> Zoom(0.9:0.1:1.2) |> CropSize(64,64)
img_processed = augment(img, pl)

I hope this makes sense. Thanks in advance.

[johnnychen94]

This seems impossible to me as this package doesn't track operations, but perhaps @Evizero has some ideas.

[Evizero]

Mhm, I have been away too long to be confident enough that I remember things correctly, but I think this should work:

In essence most operations of this package don't know and don't care what information you store in each element of the img array you pass through. Usually its pixel in form of RGB structs, but you could just as well define your own struct. Then you initialize some new pos array with that struct as eltype with the same size as img, such that each element encodes its own x,y position. You would need to define some operations on said struct so that interpolations.jl works with them as well. Another way to put it is just create your own "color" type that you use to encode the pixel position in the original array, instead of a color. And then you could do img_processed, displ_field = augment((img, pos), pl)