valentinitnelav
commented
2 years ago EDIT
This is not a bug, I was not aware of the global variables - see https://github.com/stark-t/PAI/issues/24#issuecomment-1261177484
This is obsolete now:
@stark-t , I try to debug and understand the behavior of exif_size and I run into a problem and I need your help to double-check this.
The function exif_size is defined like this in utils/dataloaders.py:
def exif_size(img):
# Returns exif-corrected PIL size
s = img.size # (width, height)
try:
rotation = dict(img._getexif().items())[orientation]
if rotation in [6, 8]: # rotation 270 or 90
s = (s[1], s[0])
except Exception:
pass
return sIt is used in verify_image_label() function in the same script like this:
im = Image.open(im_file)
im.verify() # PIL verify
shape = exif_size(im) # image sizeBut when I test it line by line with an example, the part rotation = dict(img._getexif().items())[orientation] fails with the error NameError: name 'orientation' is not defined. While that part is wrapped inside a try call, then the function never actually runs it because always gives an error and try skips it, or do I miss something obvious? This seems to me like a bug on YOLO's side.
Here is what I mean:
First, download this image on your computer, then run exif_size and get separately the image size and you get the same results despite the fact that the image is indeed rotated - the exif metadata contains "Rotate 270 CW" as posted here.
import urllib.request
import PIL
urllib.request.urlretrieve("https://observation.org/photos/2075125.jpg", "Diptera_Anthomyiidae_Delia_lamelliseta_2075125.jpg")
img_path = 'Diptera_Anthomyiidae_Delia_lamelliseta_2075125.jpg'
img = PIL.Image.open(img_path) # you have to open an image like this
exif_size(img) # first load the function from above; this gives (448, 336) It is (width, height) as a tuple
img.size # this also gives (448, 336), So, no rotation has happened despite the image having the orientation 'Rotate 270 CW"Further, the dict(img._getexif().items())[orientation] part of exif_size(img) gives an error if run directly like this:
dict(img._getexif().items())[orientation]
# NameError: name 'orientation' is not definedOne has to get the EXIF metadata as a dictionary maybe like suggested here:
import PIL.ExifTags
exif = {
PIL.ExifTags.TAGS[k]: v
for k, v in img._getexif().items()
if k in PIL.ExifTags.TAGS
}
exif['Orientation'] # only now we can index by the key 'Orientation'
# 8This looks like a bug in exif_size, because it doesn't access the EXIF metadata properly. But before reporting this on the YOLO repo, could you double check it as well?
I thought I solved this issue, but to my surprise, I still got trapped in the confusion of image orientation. There is also an open issue for YOLO here: Exif Image Orientation. Currently, the function
exif_sizewas moved toutils/dataloaders.py.I am worried that part of the errors in IoU could be due to the flip in image width with height. YOLO attempts to flip them as indicated above, when the orientation is 90 or 270 degrees. I also tried to do the same when I built the training dataset with bounding boxes, but today I got surprised that when I run some predictions, some images with 90 or 270 dg orientation were ok, but one had a flipped box and was unclear why despite trying to take care of the orientation.
So far, what I know is that the annotation tool that we use, VIA VGG can read the orientation of the image and displays it according to the orientation information from the metadata - see example here. Also, it exports the image statics as it displays it on the screen. That means that exported image statics should match how YOLO processes them by flipping them.Edit: VIA VGG "relies on the browser to load the image, according to HTML specification" - see example & discussion here. This will not always correspond to how PIL reads image width and height - see below in this thread.I think during our evaluation process, we should also extract the image statics (width, height, and orientation info) and compare that with the metadata that the VIA VGG gave when we did the annotation. I already have that but I didn't check if this matches how YOLO reads it. I will try to look into this issue.