imageio / imageio-ffmpeg

FFMPEG wrapper for Python
BSD 2-Clause "Simplified" License
233 stars 52 forks source link

Updating IMAGEIO_FFMPEG_EXE leads to `@lru_cache` related issues #75

Closed tvercaut closed 1 year ago

tvercaut commented 1 year ago

I was trying to run some imageio_ffmpeg code with different versions of the ffmpeg executable for comparison purposes.

Setting IMAGEIO_FFMPEG_EXE was unfortunately not enough and it took me a while to track down the problem. It turns out that the ffmpeg executable is being non-consistently cached through a @lru_cache decorator. It would be nice to provide a more robust means of switching ffmpeg executable from python. Either removing the use of a cache or providing a setter function for IMAGEIO_FFMPEG_EXE that takes care of clearing all relevant caches.

The culprit in my case was here: https://github.com/imageio/imageio-ffmpeg/blob/7bfe71d4a23fa881c792d23e0cf67ef4fc446dd8/imageio_ffmpeg/_io.py#L135-L137 with a workaround being to call imageio_ffmpeg._io._get_exe.cache_clear().

Steps to reproduce the problem:

# Uncomment the next line for google colab
#!pip install imageio_ffmpeg
import os
import imageio_ffmpeg
print("imageio_ffmpeg.__version__:", imageio_ffmpeg.__version__)

# The following try/catch is only relevant if this code is called several times
# e.g. in a notebook
try:
    init_ffmpeg
except NameError:
    init_ffmpeg = imageio_ffmpeg.get_ffmpeg_exe()
    init_utils_ffmpeg = imageio_ffmpeg._utils.get_ffmpeg_exe()
    init_io_ffmpeg = imageio_ffmpeg._io._get_exe()
    try:
      init_ffmpeg_version = imageio_ffmpeg.get_ffmpeg_version()
    except FileNotFoundError as e:
      print("Couldn't get ffmpeg executable version: ", e)
      init_ffmpeg_version = "not-found"

print("\ninit_ffmpeg:",init_ffmpeg)
print("init_utils_ffmpeg:",init_utils_ffmpeg)
print("init_io_ffmpeg:",init_io_ffmpeg)
print("init_ffmpeg version:",init_ffmpeg_version)

os.environ["IMAGEIO_FFMPEG_EXE"] = "ffmpeg"
print("\nSetting env IMAGEIO_FFMPEG_EXE to", os.environ["IMAGEIO_FFMPEG_EXE"])

print("imageio_ffmpeg.get_ffmpeg_exe():",imageio_ffmpeg.get_ffmpeg_exe())
print("imageio_ffmpeg._utils.get_ffmpeg_exe():",imageio_ffmpeg._utils.get_ffmpeg_exe())
print("imageio_ffmpeg._io._get_exe():",imageio_ffmpeg._io._get_exe())
imageio_ffmpeg._io._get_exe.cache_clear()
print("\nCache cleared - imageio_ffmpeg._io._get_exe():",imageio_ffmpeg._io._get_exe())
try:
  print("\nffmpeg version:",imageio_ffmpeg.get_ffmpeg_version())
except FileNotFoundError as e:
  print("\nCouldn't get ffmpeg executable version: ", e)

which leads to the following output on a current version of google colab:

imageio_ffmpeg.__version__: 0.4.7

init_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_utils_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_io_ffmpeg: /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2
init_ffmpeg version: 4.2.2-static

Setting env IMAGEIO_FFMPEG_EXE to ffmpeg
imageio_ffmpeg.get_ffmpeg_exe(): ffmpeg
imageio_ffmpeg._utils.get_ffmpeg_exe(): ffmpeg
imageio_ffmpeg._io._get_exe(): /usr/local/lib/python3.7/dist-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2

Cache cleared - imageio_ffmpeg._io._get_exe(): ffmpeg

ffmpeg version: 3.4.11-0ubuntu0.1
almarklein commented 1 year ago

I see the problem. I agree that it makes more sense if the env var was always checked. So in #77 I move the caching around to make this so.