Add HEIC support for survivor avatars

[toconnell]

Uploading HEIC (as iPhones do) bombs out during resize:

[2021-05-19 16:51:57] INFO:     [0.151776] __init__(, _id: 6011e7e1e048b77a2bf3c760 normalize_on_init: False)
[2021-05-19 16:51:57] ERROR:    PIL could not initialize an object from incoming string!
[2021-05-19 16:51:57] ERROR:    cannot identify image file <_io.BytesIO object at 0x7fd2b18850a0>
Traceback (most recent call last):
  File "/home/toconnell/kdm-manager-api/app/models/survivors/__init__.py", line 1179, in set_avatar
    im = Image.open(BytesIO(avatar))
  File "/home/toconnell/kdm-manager-api/venv/lib/python3.6/site-packages/PIL/Image.py", line 2959, in open
    "cannot identify image file %r" % (filename if filename else fp)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7fd2b18850a0>

==> logs/error.log <==
[2021-05-19 16:51:57] ERROR:    [400] PIL could not initialize an object from incoming string!
Traceback (most recent call last):
  File "/home/toconnell/kdm-manager-api/app/models/survivors/__init__.py", line 1179, in set_avatar
    im = Image.open(BytesIO(avatar))
  File "/home/toconnell/kdm-manager-api/venv/lib/python3.6/site-packages/PIL/Image.py", line 2959, in open
    "cannot identify image file %r" % (filename if filename else fp)
PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7fd2b18850a0>

[toconnell]

Looks like we're doing some scrubbing of posted IMG content in the legacy webapp. This is from session.py:

154     # scrub payload for images and encode them, if found
155     for k in payload.keys():
156         if not isinstance(payload[k], ObjectId) and payload[k] is not None:    # ignore OIDs
157             if imghdr.what(None, payload[k]) is not None:
158                 payload[k] = payload[k].encode('base64')

[toconnell]

Alright, false alarm, we're doing avatar loads purely in JS; that dogleg in the legacy session.py is vestigial and I'm going to kill it.

[toconnell]

We're using pyheif to solve for this.

Prod API has been updated:

(venv) toconnell@advanced-kdm-manager:~/kdm-manager-api$ pip install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/cd/6f/43037c7bcc8bd8ba7c9074256b1a11596daa15555808ec748048c1507f08/pip-21.1.1-py3-none-any.whl (1.5MB)
    100% |████████████████████████████████| 1.6MB 428kB/s 
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-21.1.1
(venv) toconnell@advanced-kdm-manager:~/kdm-manager-api$ pip install pyheif
Collecting pyheif
  Downloading pyheif-0.5.1-cp36-cp36m-manylinux2014_x86_64.whl (8.2 MB)
     |████████████████████████████████| 8.2 MB 4.6 MB/s 
Requirement already satisfied: cffi>=1.0.0 in ./venv/lib/python3.6/site-packages (from pyheif) (1.12.2)
Requirement already satisfied: pycparser in ./venv/lib/python3.6/site-packages (from cffi>=1.0.0->pyheif) (2.19)
Installing collected packages: pyheif
Successfully installed **pyheif-0.5.1**

[toconnell]

(venv) toconnell@mona:~/kdm-manager-api$ git diff requirements.txt
diff --git a/requirements.txt b/requirements.txt
index 2c7fa76..20cefdc 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -24,6 +24,7 @@ Pillow==8.1.1
 pkg-resources==0.0.0
 py==1.10.0
 pycparser==2.19
+pyheif==0.5.1
 PyJWT==1.7.1
 pylint==2.6.0
 pymongo==3.11.1
@@ -37,4 +38,5 @@ ua-parser==0.8.0
 urllib3==1.26.4
 user-agents==2.0
 Werkzeug==1.0.1
+whatimage==0.0.3
 wrapt==1.12.1

[toconnell]

...and the code update:

@@ -204,6 +205,16 @@ class Survivor(models.UserAsset):
             self.normalize()

+    def load(self):
+        """ Calls the base class load() method and then sets a few additional,
+        survivor-specific attributes. """
+
+        super().load()
+
+        # 2021-05-18 this is deprecated / legacy and needs to die
+        self.settlement_id = self.survivor["settlement"]
+
+
     def save(self, verbose=True):
         """ Do custom post-process for survivors and then super the base class
         func (i.e. the UserAsset.save() method) to go the generic save. """
@@ -1155,7 +1166,7 @@ class Survivor(models.UserAsset):
         # now set the type. this validates that we've got an actual image
         # in the incoming string/object
         try:
-            avatar_type = imghdr.what(None, avatar)
+            avatar_type = whatimage.identify_image(avatar)
         except TypeError:
             raise utils.InvalidUsage(
                 "Image type of incoming file could not be determined!"
@@ -1166,13 +1177,20 @@ class Survivor(models.UserAsset):

         processed_image = BytesIO()
         try:
-            im = Image.open(BytesIO(avatar))
-#            self.logger.debug('AVATAR IMAGE INITIALIZED: %s' % im)
+            # issue #50; HEIF/iPhone support
+            if avatar_type in ['heic', 'avif']:
+                heif_object = pyheif.read_heif(avatar)
+                im = Image.frombytes(
+                    mode=heif_object.mode,
+                    size=heif_object.size,
+                    data=heif_object.data
+                )
+            else:
+                im = Image.open(BytesIO(avatar))
         except Exception as e:
-            msg = "PIL could not initialize an object from incoming string!"
-            self.logger.error(msg)
+            err = "Image could not be processed! "
             self.logger.exception(e)
-            raise utils.InvalidUsage(msg)
+            raise utils.InvalidUsage(err + str(e))