helgeerbe / picframe

Picture frame viewer for raspi, controlled via mqtt and automatticly integrated as mqtt device in homeassistant.
MIT License
98 stars 28 forks source link

Problems with HEIC images even with pyheif installed #286

Closed TopDown71 closed 1 year ago

TopDown71 commented 1 year ago

Describe the bug picframe throws warning when opening HEIC images and fails if a HEIC is the first image. NOTE: username changed from 'pi' to 'myname' when building image using Raspberry Pi Imager

To Reproduce build picframe, launch from terminal window, and confirm okay with .jpg images

add HEIC images

launch from terminal window and HEIC images display but: WARNING:get_image_meta.GetImageMeta:Failed attempt to convert /home/myname/Pictures/IMG_7823.HEIC Have you installed pyheif?

Install pyheif using steps by @luksan in https://github.com/pi3d/pi3d_demos/issues/24 since steps by @vinyli85 and @sapnho fail due to a version issue with Autoconf requiring 2.71 and I can't find how to update.

launch from terminal window - images display with same warning messages.

If this doesn't cause a problem when run from systemd then it would be okay - BUT

if the HEIC image is the first displayed by picframe the result in the terminal window is: picframe ~/picframe_data/config/configuration.yaml INFO:start.py:starting ['/home/myname/.local/bin/picframe', '/home/myname/picframe_data/config/configuration.yaml'] INFO:model.Model:Open config file: /home/myname/picframe_data/config/configuration.yaml: WARNING:get_image_meta.GetImageMeta:Failed attempt to convert /home/myname/Pictures/IMG_7823.HEIC Have you installed pyheif? Traceback (most recent call last): File "/home/myname/.local/bin/picframe", line 10, in sys.exit(main()) File "/home/myname/.local/lib/python3.7/site-packages/picframe/start.py", line 160, in main c.loop() File "/home/myname/.local/lib/python3.7/site-packages/picframe/controller.py", line 321, in loop (loop_running, skip_image) = self.viewer.slideshow_is_running(pics, time_delay, fade_time, self.paused) File "/home/myname/.local/lib/python3.7/site-packages/picframe/viewer_display.py", line 496, in slideshow_is_running wh_rat = (self.display.width * self.sfg.iy) / (self.display.height * self.sfg.ix)

There is no picframe display and no exit to terminal. Pi GUI is still functional. ctrl-C in terminal flickers the GUI once and terminal displays: AttributeError: 'NoneType' object has no attribute 'iy' ^CYou pressed Ctrl-c!

Terminal window returns to normal.

Expected behavior No errors, warnings, or terminal session freeze

Screenshots see above

What are your OS, Python and picframe versions(please complete the following information):?

Checking required packages...... PIL : 9.2.0 exifread : 3.0.0 pi3d : 2.49 yaml : 6.0 paho.mqtt : 1.6.1 iptcinfo3 : 2.1.4 numpy : 1.16.2 ninepatch : installed, but no version info

Checking optional packages...... pyheif : Not found! This seems strange considering I had what looked like a successful install

$ cat ~/picframe_data/config/configuration.yaml
viewer:
  blur_amount: 12                         # default=12,  larger values than 12 will increase processing load quite a bit
  blur_zoom: 1.0                          # default=1.0, must be >= 1.0 which expands the backgorund to just fill the space around the image
  blur_edges: False                       # default=False, use blurred version of image to fill edges - will override FIT = False
  edge_alpha: 0.5                         # default=0.5, background colour at edge. 1.0 would show reflection of image
  fps: 20.0                               # default=20.0
  background: [0.2, 0.2, 0.3, 1.0]        # default=[0.2, 0.2, 0.3, 1.0], RGBA to fill edges when fitting
  blend_type: "blend"                     # default="blend", choices={"blend", "burn", "bump"}, type of blend the shader can do
  font_file: "/home/myname/picframe_data/data/fonts/NotoSans-Regular.ttf"
  shader: "/home/myname/picframe_data/data/shaders/blend_new"
  show_text_fm: "%b %d, %Y"               # default "%b %d, %Y", format to show date over the image
  show_text_tm: 15.0                      # default=20.0, time to show text over image with file name
  show_text_sz: 25                        # default=40, text character size
  show_text: "title caption name date location"  # default="title caption name date folder location", show text, include combination of words: title, caption name, date, location, folder
  text_justify: "C"                       # text justification L, C or R
  text_bkg_hgt: 0.25                      # default=0.25 (0.0-1.0), percentage of screen height for text background texture
  text_opacity: 1.0                       # default=1.0 (0.0-1.0), alpha value of text overlay
  fit: False                              # default=False, True => scale image so all visible and leave 'gaps'
                                          #                False => crop image so no 'gaps'
  kenburns: False                         # default=False, will set fit->False and blur_edges->False
  display_x: 0                            # offset from left of screen (can be negative)
  display_y: 0                            # offset from top of screen (can be negative)
  display_w: null                         # width of display surface (null->None will use max returned by hardware)
  display_h: null                         # height of display surface
  use_glx: False                          # default=False. Set to True on linux with xserver running

  mat_images: 0.01                        # default=0.01, True, automatically mat all images. False, don't automatically mat any images. Real value, auto-mat all images with aspect ratio difference > than value
  mat_type: null                          # default=null, A string containing the mat types to choose from when matting images. It can consist of any or
                                          # all of 'float float_polaroid float_color_wrap single_bevel double_bevel double_flat' (null or '' will use all mat types)
  outer_mat_color: null                   # default=null, Color of the outer mat as an RGB list. null will auto-select a reasonable color based on the image.
  inner_mat_color: null                   # default=null, Color of the inner mat as an RGB list. null will auto-select a reasonable color based on the image.
  outer_mat_border: 60                    # default=75, Minimum outer mat border in pixels
  inner_mat_border: 25                    # default=40, Minimum inner mat border in pixels (for styles that use it)
  outer_mat_use_texture: True             # default=True, True uses a texture for the outer mat. False creates a solid-color outer mat.
  inner_mat_use_texture: False            # default=False, True uses a texture for the inner mat. False creates a solid-color inner mat.
  mat_resource_folder: "/home/myname/picframe_data/data/mat" # Folder containing mat image files

  show_clock: False                       # default=False, True shows clock overlay. False does not show clock overlay
  clock_justify: "R"                      # default="R", clock justification L, C, or R
  clock_text_sz: 120                      # default=120, clock character size
  clock_format: "%I:%M"                   # default="%I:%M", strftime format for clock string
  clock_opacity: 1.0                      # default=1.0 (0.0-1.0), alpha value of clock overlay

  menu_text_sz: 40                        # default=40, menu character size
  menu_autohide_tm: 10.0                  # default=10.0, time in seconds to show menu before auto hiding (0 disables auto hiding)
  geo_suppress_list: []                   # default=None, substrings to remove from the location text

model:
  pic_dir: "/home/myname/Pictures"                   # default="/home/myname/Pictures", root folder for images
  deleted_pictures: "/home/myname/DeletedPictures"   # move deleted pictures here
  follow_links: False                     # default=False, By default, picframe will not walk down into symbolic links that resolve to directories. Set follow_links to True to visit directories pointed to by symlinks, on systems that support them.
  no_files_img: "/home/myname/picframe_data/data/no_pictures.jpg" # default="PictureFrame2020img.jpg", image to show if none selected
  subdirectory: ""                        # default="", subdir of pic_dir - can be changed by MQTT"
  recent_n: 7                             # default=7 (days), when shuffling file change date more recent than this number of days play before the rest
  reshuffle_num: 1                        # default=1, times through before reshuffling
  time_delay: 21.0                       # default=200.0, time between consecutive slide starts - can be changed by MQTT
  fade_time: 7.0                         # default=10.0, change time during which slides overlap - can be changed by MQTT"
  shuffle: True                           # default=True, shuffle on reloading image files - can be changed by MQTT"
  sort_cols: 'fname ASC'                  # default='fname ASC' can be any columns in the table with optional ASC or DESC separated by commas
                                          # fname, last_modified, file_id, orientation, exif_datetime, f_number,
                                          # exposure_time, iso, focal_length, make, model, lens, rating,
                                          # latitude, longitude, width, height, title, caption, tags,
                                          # is_portrait, location
  image_attr: [                           # image attributes send by MQTT, Keys are taken from exifread library, "PICFRAME GPS" is special to retrieve GPS lon/lat, "PICFRAME LOCATION" is special to retrieve geo reverse (load_geoloc hast to be True)
    "PICFRAME GPS",
    "PICFRAME LOCATION",
    "EXIF FNumber",
    "EXIF ExposureTime",
    "EXIF ISOSpeedRatings",
    "EXIF FocalLength",
    "EXIF DateTimeOriginal",
    "Image Model",
    "Image Make",
    "IPTC Caption/Abstract",
    "IPTC Object Name",
    "IPTC Keywords"]
  load_geoloc: False                      # get location information from open street map NB if you switch this on (recommended)
  geo_key: "this_needs_to@be_changed"     # then you **MUST** change the geo_key to something unique to you
                                          # i.e. use your email address
  locale: "en_US.UTF-8"                    # "locale -a" shows the installed locales which could used
  key_list: [
    ["tourism","amenity","isolated_dwelling"],
    ["suburb","village"],
    ["city","county"],
    ["region","state","province"],
    ["country"]]
  db_file: "/home/myname/picframe_data/data/pictureframe.db3" # database used by PictureFrame
  portrait_pairs: False
  log_level: "WARNING"                    # default=WARNING, could beDEBUG, INFO, WARNING, ERROR, CRITICAL
  log_file: ""                            # default="" for debugging set this to the path to a file. NB logging messages will
                                          # appended indefinitely so don't forget this. You will need to tidy it up later

mqtt:
  use_mqtt: False                         # default=False. Set True true, to enable mqtt
  server: "your_mqtt_broker"              # No defaults for server
  port: 8883                              # default=8883 for tls, 1883 else (tls must be "" then !!!!!)
  login: "name"                           # your mqtt user
  password: "your_password"               # password for mqtt user
  tls: "/path/to/your/ca.crt"             # filename including path to your ca.crt. If not used, must be set to "" !!!!
  device_id: "picframe"                   # default="picframe" unique id of device. change if there is more than one PictureFrame
  device_url: ""                          # if use_http==True, set url to picframe config page. Must be a valid url, or "" otherwise home assistant runs in an error.

http:
  use_http: False                         # default=False. Set True to enable http NB THIS SERVER IS FOR LOCAL NETWORK AND SHOULD NOT BE EXPOSED TO EXTERNAL ACCESS
  path: "/home/myname/picframe_data/html"            # path to where html files are located
  port: 9000                              # port used to serve pages by http server < 1024 requires root which is *bad* idea
  use_ssl: False
  keyfile: "path/to/key.pem"              # private-key
  certfile: "path/to/cert.pem"            # server certificate

peripherals:
  input_type: null                        # default=null, valid options: {null, "keyboard", "touch", "mouse"}
  buttons:
    pause:                                # pause/unpause the show
      enable: True                        # default=True
      label: "Pause"                      # default="Pause"
      shortcut: " "                       # default=" "
    display_off:                          # turn off the display (when off, any input from selected peripheral will turn it back on)
      enable: True                        # default=True
      label: "Display off"                # default="Display off"
      shortcut: "o"                       # default="o"
    location:                             # shows or hides location information
      enable: False                       # default=False
      label: "Location"                   # default="Location"
      shortcut: "l"                       # default="l"
    exit:                                 # exit PictureFrame
      enable: False                       # default=False
      label: "Exit"                       # default="Exit"
      shortcut: "e"                       # default="e"
    power_down:                           # power down the device, uses sudo
      enable: False                       # default=False
      label: "Power down"                 # default="Power down"
      shortcut: "p"                       # default="p"
log output goes here

Additional context Add any other context about the problem here.

paddywwoof commented 1 year ago

Hmm. Firstly thanks for that super explicit bug report. I remember HEIC being a bit of a nightmare to install so I suspect your problems are linked to whatever prevented you from following @sapnho steps. I have only quickly read your notes on my phone (I'm away from home at the moment) but do you have to use python 2?

TopDown71 commented 1 year ago

Thanks for the quick reply! I’ve been ‘hacking’ code for about 44 years now - mainly as a way to get more done in my job. So it’s very much utility FORTRAN and VBA with a scattering of C. I’m somewhat familiar with python, but don’t get into it all that often. My history and trying to fix things for people I shared code with explains my level of detail 😀

I just typed python —version so I guess it defaulted to 2. I do have python 3.7.3 on the Pi as well. I guess it’s part of the standard Raspberry PI Imager build unless it came along for the ride with the picframe install.

My Pi shows no updates or upgrades available so I don’t see how to update autoconf. When I check the version I get:

apt show autoconf Package: autoconf Version: 2.69-11 Priority: optional Section: devel Maintainer: Ben Pfaff @.***> Installed-Size: 1,913 kB Depends: perl (>> 5.005), m4 (>= 1.4.13), debianutils (>= 1.8) Recommends: automake | automaken Suggests: autoconf-archive, gnu-standards, autoconf-doc, libtool, gettext Breaks: gettext (<< 0.10.39), pkg-config (<< 0.25-1.1) Homepage: http://www.gnu.org/software/autoconf/ Download-Size: 341 kB APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages Description: automatic configure script builder The standard for FSF source packages. This is only useful if you write your own programs or if you extensively modify other people's programs. . For an extensive library of additional Autoconf macros, install the `autoconf-archive' package. . This version of autoconf is not compatible with scripts meant for Autoconf 2.13 or earlier.

On Oct 24, 2022, at 9:37 AM, paddywwoof @.***> wrote:

Hmm. Firstly thanks for that super explicit bug report. I remember HEIC being a bit of a nightmare to install so I suspect your problems are linked to whatever prevented you from following @sapnho https://github.com/sapnho steps. I have only quickly read your notes on my phone (I'm away from home at the moment) but do you have to use python 2?

— Reply to this email directly, view it on GitHub https://github.com/helgeerbe/picframe/issues/286#issuecomment-1289138586, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG6TDYQ26O6BUD5RPOUNG53WE2NLPANCNFSM6AAAAAARM62JFY. You are receiving this because you authored the thread.

bigcat88 commented 1 year ago

maybe a bit brazen, but wouldn't it be easier to just use pillow_heif instead of pyheif?

Pros, to do that:

  1. It has working arm7 wheels(that will probably work on Rasberry).
  2. It supports EXIF and other standard Pillow's meta data.
  3. It is simplier to use, just adding it as a Pillow plugin.
paddywwoof commented 1 year ago

@bigcat88 has a good point. I think pillow_heif was in an early state of development or didn't work when the how-to and module were written. It could well be that they would be a tidier approach now.

helgeerbe commented 1 year ago

@bigcat88 Thanks for the hint. If I have some time left, I will give it a try. Looks like it could easily integrated.

TopDown71 commented 1 year ago

Thank you both for looking into this! As a followup, I was trying to confirm the heic image displayed, but it does not. The heic file name and date appear, but picframe displays what must be the next displayable image. If it's just this small bit of code that has to be touched it's worth a shot.

helgeerbe commented 1 year ago

I created a branch pi_heif and pulled #288 from @bigcat88 . I did some changes to the original pr.

On my Mac, everything seems to work fine. I will merge it to dev for others to test.

bigcat88 commented 1 year ago

Just a note: you can call register_heif_opener once anywhere in project. For example in start.py, and it will be enougth. Just to simplify code even more :)

helgeerbe commented 1 year ago

I tried that. But get_image_object(fname): is a static function. I have to call register_heif_opener() in its context.

helgeerbe commented 1 year ago

Actual I have some issues with the meta data. They are read properly, but don't show up on home assistant. I have to find out why.

helgeerbe commented 1 year ago

There seems to be a problem with exifread lib 3.0.0 on raspberry (https://github.com/ianare/exif-py/issues/160). Downgrading to a version < 3 solves the problem.

helgeerbe commented 1 year ago

@bigcat88 I think about to remove exifread lib from picframe, with your lib Pillow supports heic/heif images.

I wrote a small test to compare the values for a heic image with the exifread tool. And I found that I got the wrong orientation. 1 instead of 6. The rest looks fine. Any ideas?

import pytest
import logging

from PIL import Image
from PIL.Image import Exif
from PIL.ExifTags import TAGS, GPSTAGS
from pi_heif import register_heif_opener
from fractions import Fraction

logger = logging.getLogger("test_get_exifs_by_pillow")
logger.setLevel(logging.DEBUG)

register_heif_opener()

def get_labeled_exif(exif):
    return {
        TAGS.get(key, key): value
        for key, value in exif.items()
    }

def get_exif_ifd(exif):
    for key, value in TAGS.items():
        if value == "ExifOffset":
            break
    info = exif.get_ifd(key)
    return {
        TAGS.get(key, key): value
        for key, value in info.items()
    }

def get_geo(exif):
    for key, value in TAGS.items():
        if value == "GPSInfo":
            break
    gps_info = exif.get_ifd(key)
    return {
        GPSTAGS.get(key, key): value
        for key, value in gps_info.items()
    }

def convert_to_degrees(value):
    (deg, min, sec) = value
    d = float(deg.numerator) / float(deg.denominator if deg.denominator > 0 else 1) #TODO better catching?
    m = float(min.numerator) / float(min.denominator if min.denominator > 0 else 1)
    s = float(sec.numerator) / float(sec.denominator if sec.denominator > 0 else 1)
    return d + (m / 60.0) + (s / 3600.0)

def test_pillow_exifs_heic():
    try:
        image = Image.open("test/images/test3.HEIC")

        exifs = image.getexif()
        labled_exif = get_labeled_exif(exifs)
        labled_exif_ifd = get_exif_ifd(exifs)
        labled_exif_geo = get_geo(exifs)

        print(labled_exif)
        print(labled_exif_ifd)
        print(labled_exif_geo)

        orientation = labled_exif['Orientation']
        assert  orientation == 6
        assert labled_exif['Make'] == "Apple"
        assert labled_exif['Model']  == "iPhone 8"

        assert labled_exif_ifd['ExifImageWidth'] == 4032
        assert labled_exif_ifd['ExifImageHeight'] == 3024
        assert labled_exif_ifd['FNumber']== 1.8
        assert str(Fraction(labled_exif_ifd['ExposureTime']))== "1/5"
        assert labled_exif_ifd['ISOSpeedRatings']== 100
        assert labled_exif_ifd['FocalLength']== 3.99
        assert labled_exif_ifd['LensModel']== "iPhone 8 back camera 3.99mm f/1.8"
        assert labled_exif_ifd['DateTimeOriginal']== "2021:05:14 20:27:14"

        # GPS
        lat = convert_to_degrees(labled_exif_geo['GPSLatitude'])
        if len(labled_exif_geo['GPSLatitudeRef']) > 0 and labled_exif_geo['GPSLatitudeRef'][0] == 'S':
            # assume zero length string means N
            lat = 0 - lat
        assert lat == 38.71365

        lon = convert_to_degrees(labled_exif_geo['GPSLongitude'])
        if len(labled_exif_geo['GPSLongitudeRef']) > 0 and labled_exif_geo['GPSLongitudeRef'][0] == 'W':
            # assume zero length string means N
            lon = 0 - lon
        assert lon == -78.15960555555556

    except:
        pytest.fail("Unexpected exception")
bigcat88 commented 1 year ago

The problem with orientation is described in docs. It is a complex problem for HEIF... As for this usage when you only decodes images, i can suggest ignore orientation for HEIF files at all. Libheif currently do not expose irot values, which indicates orientation. And for Exif-XMP orientation different software behaves differently, some of them rotate HEIF images, some not(which is ok for HEIF, rotation in HEIF specified in different place, and should be ignored in EXIF)

helgeerbe commented 1 year ago

@bigcat88 Thanks for the hint. I removed also the "exifread" lib. Will test this dev branch on my frame. If everything works well I will make a new release, which supports heif/heic natively.

TopDown71 commented 1 year ago

Thank you both for following up on what was obviously more than just a super simple issue. I’ll install a new version when I see that it is available.

Regards, Dan

On Nov 30, 2022, at 10:21 AM, helgeerbe @.***> wrote:

@bigcat88 https://github.com/bigcat88 Thanks for the hint. I removed also the "exifread" lib. Will test this dev branch on my frame. If everything works well I will make a new release, which supports heif/heic natively.

— Reply to this email directly, view it on GitHub https://github.com/helgeerbe/picframe/issues/286#issuecomment-1332420061, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG6TDYRXXDD2YFCYIVSE7O3WK55KHANCNFSM6AAAAAARM62JFY. You are receiving this because you authored the thread.

helgeerbe commented 1 year ago

Should be fixed with release 2022.12.04

TopDown71 commented 1 year ago

I had a family situation come up over the holidays and finally got to build a new frame over the last few days. Everything seems to be working perfectly now. Thanks for all of the joint effort it took to get there!

Happy new year and kind regards

On Oct 24, 2022, at 3:20 PM, Alexander Piskun @.***> wrote:

maybe a bit brazen, but wouldn't it be easier to just use pillow_heif instead of pyheif?

Pros, to do that:

It has working arm7 wheels(that will probably work on Rasberry). It supports EXIF and other standard Pillow's meta data. It is simplier to use, just adding it as a Pillow plugin. — Reply to this email directly, view it on GitHub https://github.com/helgeerbe/picframe/issues/286#issuecomment-1289564760, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG6TDYVF5KGOLGWN46OVXOLWE3VRJANCNFSM6AAAAAARM62JFY. You are receiving this because you authored the thread.