bin/im2rec rotating images 90 degrees counter-clockwise.

[jrosebr1]

I am using the bin/im2rec tool to generate a .rec file for the Kaggle Dogs vs. Cats image challenge. I used the following code to visualize the output of ImageRecordIter:

# construct the training image iterator
trainIter = mx.io.ImageRecordIter(
    path_imgrec=config.TEST_MX_REC,
    data_shape=(3, 227, 227),
    batch_size=128 * 3,
    rand_crop=True,
    rand_mirror=True,
    preprocess_threads=4)

# grab the first batch of images from the generator
images = trainIter.getdata().asnumpy()

# loop over a few images
for i in np.arange(0, 100):
    # grab the image and transpose it to RGB ordering
    image = images[i].transpose((0, 2, 1)).astype("uint8")
    (R, G, B) = (image[0], image[1], image[2])
    image = np.dstack([R, G, B])

    # write the image to file
    p = [args["output"], "{}.png".format(i + 1)]
    io.imsave(os.path.sep.join(p), image)

However, when I inspected the output files, all of the images were rotated exactly 90 degrees counter-clockwise.

I have attached a few of the examples to this note. I'm just curious if I am doing something wrong or if this is a bug in the im2rec binary?

[jrosebr1]

I added a few cv2.imshow calls to the tools/im2rec.py to see if I could debug that is going on. I placed a call right after cv2.imshow and then another right before the image is being written to pack_img. As the attachment shows, the image it's not an OpenCV issue.

Either the image is being rotated as it's being read out of the .rec file or the cv2.imencode call is doing something strange. What's more likely is that it's an issue when the image is read back out. I'll see if I can look into it more.

Note: I tried creating the .rec file with both bin/im2rec and tools/im2rec and it was the same 90 degree counter-clockwise result each time.

[jrosebr1]

Another update: I tried using bin/im2rec on a separate dataset (ImageNet) and I'm still seeing the 90 degree rotation issue. Still investigating what the cause is.

[jrosebr1]

More updates. I defined the following function in tools/im2rec.py to handle rotating an image while ensuring parts of it are not cut off:

def rotate_bound(image, angle):
    import numpy as np

    # grab the dimensions of the image and then determine the center
    (h, w) = image.shape[:2]
    (cX, cY) = (w // 2, h // 2)

    # grab the rotation matrix (applying the negative of the angle
    # to rotate clockwise), then grab the sine and cosine (i.e.,
    # the rotation components of the matrix)
    M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
    cos = np.abs(M[0, 0])
    sin = np.abs(M[0, 1])

    # compute the new bounding dimensions of the image
    nW = int((h * sin) + (w * cos))
    nH = int((h * cos) + (w * sin))

    # adjust the rotation matrix to take into account translation
    M[0, 2] += (nW / 2) - cX
    M[1, 2] += (nH / 2) - cY

    # perform the actual rotation and return the image
    return cv2.warpAffine(image, M, (nW, nH))

This function will soon to be part of my imutils package.

I then modified image_encode such that:

    if args.resize:
        if img.shape[0] > img.shape[1]:
            newsize = (args.resize, img.shape[0] * args.resize / img.shape[1])
        else:
            newsize = (img.shape[1] * args.resize / img.shape[0], args.resize)
        img = cv2.resize(img, newsize)

        img = rotate_bound(img, -90)

Now when I read images back out of the ImageRecordIter they are correctly oriented. However, this is a hack and still does imply there is an issue with how either ImageRecordIter is storing or retrieving the images.

But there still is another problem: rand_mirror=True now flips images vertically instead of horizontally. I think this needs to be addressed in the ImageRecordIter code. I'll start diving into that.

[jrosebr1]

@piiswrong @tornadomeet Are these issues related to the recent fix that were pushed for #3975?

[jrosebr1]

I think it was the NumPy transpose that was throwing me off. This seems to work:

# grab the first batch of images from the generator
images = trainIter.getdata().asnumpy()

# loop over a few images
for i in np.arange(0, 10):
    # grab the image and transpose it to RGB ordering
    image = images[i]
    (R, G, B) = (image[0], image[1], image[2])
    image = np.dstack([B, G, R]).astype("uint8")

    cv2.imshow("Image", image)
    cv2.waitKey(0)

[yajiedesign]

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!