Results vary greatly between setting IMAGE_RESIZE_MODE = "square" and "none"

[MEIXuYan]

Hello, I trained maskrcnn model with my image datasets.When I set IMAGE_RESIZE_MODE = "square", it works well , get mAP(iou=50%)=0.817. BUT when I tried training networks with origion size images by setting IMAGE_RESIZE_MODE = "none", I got very poor evaluation mAP(iou=50%)=0.15. I used tensorboard and noticed that the loss behave unnormal(IMAGE_RESIZE_MODE="none"):

detail about my dataset: 1 All image has same size 640x480 2 Only two classes: "background" and "object", but many images contain multi instances

Does anyone know why this happen?

[zungam]

none: No resizing. Return the image unchanged.
        square: Resize and pad with zeros to get a square image
            of size [max_dim, max_dim].
        pad64: Pads width and height with zeros to make them multiples of 64.
               If min_dim or min_scale are provided, it scales the image up
               before padding. max_dim is ignored in this mode.
               The multiple of 64 is needed to ensure smooth scaling of feature
               maps up and down the 6 levels of the FPN pyramid (2**6=64).
        crop: Picks random crops from the image. First, scales the image based
              on min_dim and min_scale, then picks a random crop of
              size min_dim x min_dim. Can be used in training only.
              max_dim is not used in this mode.

I dont know what is going on here, but wouldnt recomend using none. The image need to be resized to fit your input into the neural network, which is a square.
If you send in 640x480 images and use None, they are not resized to a square, so i presume nothing starts working

[MEIXuYan]

Thank you for your advice@zungam When I trained the network with IMAGE_RESIZE_MODE = "none", no bug came up only the evaluation were poor. I have tried FCN based methods which did not need to resize the input image, and for proposal based methods such as maskrcnn I can understand it need the inpurt image to be in same size, but why it need to be square? Dose RPN or other part of the architecture needs square image to generate good results? Dose anyone konw? thanks.

[nguyenha1121]

Hey @MEIXuYan can u share your training schedule numbers of images and size image? I have same issue as you and i think my model have overfitting probleme. Thank you!

[ApoorvaSuresh]

I work with image_resize_mode = crop, and it works fine. Does it work well with mode = none ?

[jasdal365]

With mode "none" the loss is half of mode "square", but all losses containing "mrcnn" don't work anymore. Does anyone's got the same problem?