search

kieraneglin / pinchflat

Your next YouTube media manager
GNU Affero General Public License v3.0
1.42k stars 25 forks source link

[Triage] ERROR: Preprocessing: ffmpeg not found. #347

Closed Snake883 closed 3 months ago

Snake883 commented 3 months ago

Describe the bug

[error] yt-dlp download error for media item #509: "ERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location\nERROR: Preprocessing: ffmpeg not found.

To Reproduce I assume this might have something with me wanting to embed thumbnails and metadata into the media file, but not sure.

Diagnostic info

Additional context

Docker

I also ran ffmpeg from the docker container:

I have no name!@c1fc5bc3c397:/bin$ /bin/ffmpeg 
Illegal instruction
Snake883 commented 3 months ago

I opened the docker container and manually installed ffmpeg: # apt-get install ffmpeg

I then tried to force downloading YT video, and nothing happens, but at least I no longer get an error message.

kieraneglin commented 3 months ago

Hey there!

This doesn't actually sound like an issue with ffmpeg, this sounds like an issue with the container itself (somehow) or your host operating system.

I'd recommend you delete the current container completely and try re-installing fresh. There aren't any other reports of this so I'm confident it's not a widespread issue, which means it must be a one-off fluke or something isolated to your setup. Usually a nuke + reinstall works for stuff like this!

Out of curiosity, what's the Docker container URL you're using? Is it ghcr.io/kieraneglin/pinchflat:latest?

Snake883 commented 3 months ago

I deleted the container and the image, and did a fresh reinstall (repull from ghcr.io/kieraneglin/pinchflat:latest repository) from UnRaid Community App.

Problem still exists.

Curious if it might have something to do with https://github.com/kieraneglin/pinchflat/pull/231.

kieraneglin commented 3 months ago

I'll keep looking into this, but I really suspect it's something unique to your setup - either a Docker virtualization issue or a permissions issue. My reason for thinking this is that there's been ~30,000 installs since I pushed that change and this is the only time I've heard about this issue.

Again, I'll still look into it and let you know if I find anything!

kieraneglin commented 3 months ago

I've looked back into this and it seems like the issue really is what it says on the tin - ffmpeg tried to execute an instruction that your CPU doesn't support.

The question is then what CPU do you have?

AlexFullmoon commented 3 months ago

Same error here. Running on quite ancient AMD Turion II Neo N54L (HP Microserver G7), so yes, it is probable it's missing some instruction set.

Unfortunately even with everything in media profile disabled (aside from video size set to 4k, but that should ensure no transcoding necessary, right?) it still fails.

2024-08-16 12:41:50.390 [info] | [command_wrapper]: /usr/local/bin/yt-dlp called with: https://www.youtube.com/watch?v=57j1tKFA3hI --no-simulate --skip-download --write-thumbnail --convert-thumbnail jpg --output /config/metadata/media_items/9/thumbnail.%(ext)s --print-to-file after_move:%()j /tmp/pinchflat/data/8b80c047532ada91cee7a931b001d94f574ab57872951957616f67d9212cc56d.json --windows-filenames --quiet --cache-dir /tmp/pinchflat/data/yt-dlp-cache

2024-08-16 12:41:59.022 [error] | [command_wrapper]: /usr/local/bin/yt-dlp called with: https://www.youtube.com/watch?v=57j1tKFA3hI --no-simulate --skip-download --write-thumbnail --convert-thumbnail jpg --output /config/metadata/media_items/9/thumbnail.%(ext)s --print-to-file after_move:%()j /tmp/pinchflat/data/8b80c047532ada91cee7a931b001d94f574ab57872951957616f67d9212cc56d.json --windows-filenames --quiet --cache-dir /tmp/pinchflat/data/yt-dlp-cache exited: 1 with: WARNING: ffmpeg not found. The downloaded format may not be the best available. Installing ffmpeg is strongly recommended: https://github.com/yt-dlp/yt-dlp#dependencies

ERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location

ERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location

2024-08-16 12:41:59.026 [info] | {"error":"** (Protocol.UndefinedError) protocol String.Chars not implemented for #Ecto.Changeset<action: :update, changes: %{metadata: #Ecto.Changeset<action: :insert, changes: %{metadata_filepath: \"/config/metadata/media_items/9/metadata.json.gz\"}, errors: [thumbnail_filepath: {\"can't be blank\", [validation: :required]}], data: #Pinchflat.Metadata.MediaMetadata<>, valid?: false>, media_filepath: \"/downloads/Food/Рецепт латте Чизкейк [57j1tKFA3hI].mp4\", media_downloaded_at: ~U[2024-08-16 09:41:50Z], nfo_filepath: \"/downloads/Food/Рецепт латте Чизкейк [57j1tKFA3hI].nfo\", playlist_index: 0}, errors: [], data: #Pinchflat.Media.MediaItem<>, valid?: false> of type Ecto.Changeset (a struct). This protocol is implemented for the following type(s): Atom, BitString, Date, DateTime, Decimal, Exqlite.Query, Float, Integer, List, NaiveDateTime, Phoenix.LiveComponent.CID, Time, URI, Version, Version.Requirement","args":{"force":true,"id":9},"id":13,"meta":{},"state":"failure","max_attempts":20,"queue":"media_fetching","worker":"Pinchflat.Downloading.MediaDownloadWorker","source":"oban","duration":17695737,"event":"job:exception","queue_time":36752,"attempt":5,"tags":["media_item","media_fetching","show_in_dashboard"]}

UPD: it actually downloads both video files and subtitles/thumbnails/etc. Pinchflat apparently just doesn't register it due to yt-dlp error.

kieraneglin commented 3 months ago

This is definitely an upstream issue with yt-dlp and its custom ffmpeg builds, but I want to see if re-installing will fix things.

Would you mind exec-ing into your docker container and running this:

su
apt-get update
apt-get install -y build-essential
curl -L "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz" --output /tmp/ffmpeg.tar.xz
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffmpeg"
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffprobe"

And seeing if that fixes things?

AlexFullmoon commented 3 months ago

Hmm. Same, Files are downloaded, but yt-dlp fails with ffmpeg error.

2024-08-20 22:34:47.730 [error] | [command_wrapper]: /usr/local/bin/yt-dlp called with: https://www.youtube.com/watch?v=SLhq9dmOwno --no-simulate --no-progress --force-overwrites --parse-metadata %(upload_date>%Y-%m-%d)s:(?P<meta_date>.+) --write-subs --convert-subs srt --embed-subs --sub-langs en --write-thumbnail --convert-thumbnail jpg --output thumbnail:/downloads/TestPlaylist/%(upload_date>%Y-%m-%d)S %(title)S/%(title)S [%(id)S]-thumb.%(ext)S --embed-thumbnail --convert-thumbnail jpg --embed-metadata --remux-video mp4 --format-sort res:1080,+codec:avc:m4a --output /downloads/TestPlaylist/%(upload_date>%Y-%m-%d)S %(title)S/%(title)S [%(id)S].%(ext)S --print-to-file after_move:%()j /tmp/pinchflat/data/a7e0a791b80bcf07eeed8a4309d5567bf32b7dae0048f99eddfac02f5e082671.json --windows-filenames --quiet --cache-dir /tmp/pinchflat/data/yt-dlp-cache exited: 1 with: WARNING: ffmpeg not found. The downloaded format may not be the best available. Installing ffmpeg is strongly recommended: https://github.com/yt-dlp/yt-dlp#dependencies
ERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location
ERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location
ERROR: Postprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location
2024-08-20 22:34:47.731 [error] | yt-dlp download error for media item #1: "WARNING: ffmpeg not found. The downloaded format may not be the best available. Installing ffmpeg is strongly recommended: https://github.com/yt-dlp/yt-dlp#dependencies\nERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location\nERROR: Preprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location\nERROR: Postprocessing: ffmpeg not found. Please install or provide the path using --ffmpeg-location\n"

And yes, you were correct from the start (at least for my case). Running ffmpeg directly gives Illegal instruction (core dumped).

Functional ffmpeg on host system (Synology OS) is:

ffmpeg6 --version
ffmpeg version 6.0.1-3 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 8.5.0 (GCC)
  configuration: --target-os=linux --cross-prefix=/github/workspace/toolchain/syno-x64-7.1/work/x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu- --prefix=/var/packages/ffmpeg6/target --extra-cflags=-I/github/workspace/spk/ffmpeg6/work-x64-7.1/install/var/packages/ffmpeg6/target/include --extra-ldflags=-L/github/workspace/spk/ffmpeg6/work-x64-7.1/install/var/packages/ffmpeg6/target/lib --extra-libs='-lxml2 -ldl -lm' --pkg-config=/usr/bin/pkg-config --ranlib=/github/workspace/toolchain/syno-x64-7.1/work/x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu-ranlib --enable-cross-compile --enable-rpath --enable-pic --enable-shared --enable-gpl --enable-version3 --disable-debug --disable-static --disable-doc --extra-version=3 --extra-cflags=-DSYNO_VIDEOSTATION --extra-cflags=-fno-if-conversion --extra-cflags=-O3 --extra-cflags=-Wno-deprecated-declarations --x86asmexe=nasm --enable-libcodec2 --enable-libxml2 --enable-demuxer=dash --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libopenjpeg --enable-libmp3lame --enable-libbluray --enable-libspeex --enable-libtheora --enable-libcaca --enable-libdc1394 --enable-libvorbis --enable-libwebp --enable-libzmq --enable-gnutls --enable-libopenh264 --enable-libopus --enable-libsoxr --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librabbitmq --enable-libtwolame --enable-libzvbi --enable-libx264 --enable-libx265 --enable-libvpx --enable-libshine --enable-chromaprint --enable-libdav1d --enable-librist --enable-libzimg --enable-libfdk-aac --enable-nonfree --enable-libaom --enable-libsvtav1 --enable-libsvthevc --arch=x86_64 --enable-libdrm --enable-vaapi --enable-libass --enable-frei0r --enable-libvpl
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
kieraneglin commented 3 months ago

Before I throw in the towel on this, would anyone be willing to help me try an older version of FFMPEG? This is a build from a few months ago:

su
apt-get update
apt-get install -y build-essential
curl -L "https://github.com/yt-dlp/FFmpeg-Builds/releases/download/autobuild-2024-06-30-14-06/ffmpeg-N-116058-g2902ed25b5-linux64-gpl.tar.xz" --output /tmp/ffmpeg.tar.xz
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffmpeg"
tar -xf /tmp/ffmpeg.tar.xz --strip-components=2 --no-anchored -C /usr/bin/ "ffprobe"

Truly, this is about all I can do on my end unless this is a transient breakage that a future version of FFMPEG will fix

AlexFullmoon commented 3 months ago

2024-06-30 version does work!

I'll try going upwards and check more recent builds.

kieraneglin commented 3 months ago

I have a feeling that this one may work also: https://github.com/yt-dlp/FFmpeg-Builds/releases/download/autobuild-2024-08-06-14-10/ffmpeg-N-116541-g5dfc0cc841-linux64-gpl.tar.xz

Would you be able to test that? if it works, I'll pin that version in the Dockerfile

AlexFullmoon commented 3 months ago

2024-07-30 is the latest working one.

I should set up a reminder to check future versions in a month or two. Though since it fails due to missing CPU instructions...

kieraneglin commented 3 months ago

Thank you very much for checking! I really appreciate it since I can't reproduce the issue myself.

I'll look into that version, as well as the ffmpeg issue tracker to see if I can figure out what the deal is. I'll also likely pin that version in the Dockerfile

kieraneglin commented 3 months ago

This will be fixed in the next release (later today or tomorrow)!

AlexFullmoon commented 1 month ago

Note: latest (2024-10-10) build still fails.