RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

geyang commented 3 years ago

Just regular call to imageio, and it works during the same runtime. But suddenly, this stops working and gives the following error:

Traceback (most recent call last):
  File "/Users/ge/mit/dmc_gen/dmc_gen_analysis/__init__.py", line 200, in thunk
  File "/home/gridsan/geyang/jaynes-mount/dmc_gen/2021-03-20/113123.645983/dmc_gen/dmc_gen/train.py", line 100, in train
    evaluate(env, agent, Args.eval_episodes, save_video=f"videos/{step:08d}_train.mp4")
  File "/home/gridsan/geyang/jaynes-mount/dmc_gen/2021-03-20/113123.645983/dmc_gen/dmc_gen/train.py", line 31, in evaluate
    logger.save_video(frames, key=save_video)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/ml_logger/ml_logger.py", line 1147, in save_video
    imageio.mimsave(ntp.name, img_as_ubyte(frame_stack), format=format, fps=fps, **imageio_kwargs)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/core/functions.py", line 418, in mimwrite
    writer.append_data(im)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/core/format.py", line 502, in append_data
    return self._append_data(im, total_meta)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 561, in _append_data
    self._initialize()
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 619, in _initialize
    self._write_gen.send(None)
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_io.py", line 377, in write_frames
    cmd = [_get_exe(), "-y", "-f", "rawvideo", "-vcodec", "rawvideo", "-s", sizestr]
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_io.py", line 19, in _get_exe
    exe = get_ffmpeg_exe()
  File "/home/gridsan/geyang/.conda/envs/dmcgen/lib/python3.6/site-packages/imageio_ffmpeg/_utils.py", line 50, in get_ffmpeg_exe
    "No ffmpeg exe could be found. Install ffmpeg on your system, "
RuntimeError: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Actual Behavior

This is not the first time it is called in this RunTime. For background, this is running on an HPC system. The first time this happened I also set the "IMAGEIO_FFMPEG_EXE" variable. Now I always set that variable.

The real red flag is that this function call is okay in these runs before this error is suddenly gets raised.

Steps to Reproduce the Problem

This is hardware-specific. And it takes a while to reproduce inside a ML training loop. These are the constrains I have no control over.

Specifications

Python Version: 3.6.9

ImageIO Version:

imageio                 2.9.0
imageio-ffmpeg          0.4.3

Platform Name: Linux 4.14.224-llgrid-10ms
Platform Version: Linux 4.14.224-llgrid-10ms

almarklein commented 3 years ago

Do I understand correctly that things are ok when you set IMAGEIO_FFMPEG_EXE, but are mostly worried that this error seems to occur midway?

Anyway, the only thing I can come up with right now is that the PATH environment variable is changed, causing ffmpeg to be found at first, but fails to do so later. Not much else to go on, frankly.

geyang commented 3 years ago

I did not know that path variable can be changed mid way 🤣

On Sun, Mar 21, 2021 at 09:25 Almar Klein @.***> wrote:

Do I understand correctly that things are ok when you set IMAGEIO_FFMPEG_EXE, but are mostly worried that this error seems to occur midway?

Anyway, the only thing I can come up with right now is that the PATH environment variable is changed, causing ffmpeg to be found at first, but fails to do so later. Not much else to go on, frankly.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/imageio/imageio-ffmpeg/issues/48#issuecomment-803579472, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEZ5WRO5WBO7XTIGFSMJCTTEXXUNANCNFSM4ZQYDPHA .

hmaarrfk commented 1 year ago

Just jumping on here even though my help has not been requested. But my guess is that:

Your ML training loop is eating your RAM and growing in an unbound way.
Eventually, the "test" to check that the ffmpeg executable exists is failing to spawn a process because there is no RAM in your machine anymore.

emjay73 commented 1 year ago

I got a hint from @hmaarrfk and tried os.environ["OMP_NUM_THREADS"]=1 and this solved my problem. Thanks.

imageio / imageio-ffmpeg