victorca25 / traiNNer

traiNNer: Deep learning framework for image and video super-resolution, restoration and image-to-image translation, for training and testing.
Apache License 2.0
288 stars 40 forks source link

`nearest_aligned` is not aligned #62

Open awused opened 2 years ago

awused commented 2 years ago

When using nearest_aligned the output is noticeably shifted down and to the right. This affects my models severely and causes noticeable warping in their output.

I used this code in augmentations.py to produce the output images:

if __name__ == '__main__':
    img = cv2.imread('test.png')
    img_A, _ = Scale(img=img, scale=4, algo=997, ds_kernel=None, img_type='cv2')
    cv2.imwrite('output.png', img_A)

original image

Output from nearest_aligned as per the above code: output

Output from convert test.png -interpolate Average -filter point -resize 25% magick-nearest.png: magick-nearest

Explicitly sampling the top left corner closely matches the offset from nearest_aligned: convert test.png -define sample:offset=0%x0% -sample 25% magick-sampled-top-left.png magick-sampled-top-left

victorca25 commented 2 years ago

Any change in nearest_aligned would go in https://github.com/victorca25/augmennt, but can you test what I commented here: https://github.com/victorca25/traiNNer/blob/12d006fd44ed304e4178839c53b1f3d95ca25dcb/codes/dataops/augmennt/augmennt/transforms.py#L2582

For the cases I tested, the images were aligned, but maybe in some cases it needs a larger blur kernel.

awused commented 2 years ago

I've tested it on multiple files, it's not like the input file matters. I first noticed it as warping in the output of my models, which was obviously apparent and corrected after I implemented my own local fix, but my local fix is a hack that can't be merged that only works locally because I don't trigger any other code paths that use sample().

As the code is now it's simply wrong. I cannot guess how you tested it to make it appear correct, but my tests are repeatable at both 2x and 4x.