Some images that contributors send us are not correctly oriented (they may be rotated to the left, to the right or top-down). Using Google Cloud Vision, we store the orientation of each detected word (up, right, down, left) as a prediction, in the prediction DB table, with the image_orientation type. prediction.data has the following structure:

{"count":{"up":4,"left":1,"right":18},"rotation":270,"orientation":"right"}

The count field gives you how many words were correctly oriented ("up"), left-oriented ("left"), right-oriented ("right"), or flipped ("down"). The orientation with the highest count determines the values for the remaining fields:

orientation gives you the predicted image orientation
rotation gives you the rotation to apply to get a correctly oriented image

We would like to automatically rotate incorrectly-rotated images if we're confident enough that the rotation is incorrect:

predicted orientation is not up
fraction of words with the predicted orientation >= 0.95

Steps

Create a ImageOrientationImporter class in importer.py to import them as insights
Add annotator in annotate.py

openfoodfacts / robotoff

Auto-rotate images if necessary #971

Steps