Closed EMG70 closed 1 month ago
@EMG70 can you include both?
Errors from pastebin Lines 1016-1290 below:
Related:
I've abbreviated @EMG70's answers above, to simplify testing:
- The complete yt-dlp command you used to download the video
yt-dlp https://youtu.be/RMM5N63d2DI
- The exact filename of the video (on your local machine) resulting from above
Numbers in the Teens They start with a 1!!!! song clipnabber com [RMM5N63d2DI].mp4
(4,347,392 bytes)
Which other kinds of upload testing are most important?
Cannot reproduce the issue downloading the single video
@EMG70, I successfully downloaded this video in my new VM. Does it work for you now?
@EMG70, I successfully downloaded this video in my new VM. Does it work for you now?
@deldesir do you mean "successfully uploaded" ?
(This issue is not about downloading.)
Ah I see. I will do upload check then, sorry.
I reproduced the error. Working on it now...
The exclamation marks are not the issue. I removed the exclamation marks from the title, the same issue happened.
I renamed another video after the title of the failed video (with exclamation marks included), no effect.
Verdict: ffmpeg could not generate a cover for this specific video.
@deldesir can you please paste in FFmpeg's error, after running it on @EMG70's video at the command-line?
Goal: Capture FFmpeg's error message and/or warning on this ticket, to illuminate very concrete details, Thanks!
PS Use the exact same FFmpeg command-line flags/options/parameters as IIAB Calibre-Web's uploader.py below (if possible!)
Here it is:
root@lrn2:/library/calibre-web/casnelaine/Numbers in the Teens They start with a 1!!!! song clipnabber com (1085)# ffmpeg -i Numbers\ in\ the\ Teens\ They\ start\ with\ a\ 1\!\!\ -\ casnelaine.mp4 -vf "fps=1,thumbnail,select=gt(scene\,0.1),scale=-1:720" -frames:v 1 -vsync vfr -y output_cover.jpg
ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)
configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
-vsync is deprecated. Use -fps_mode
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Numbers in the Teens They start with a 1!! - casnelaine.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Numbers in the Teens They start with a 1!!!! song clipnabber com
artist : casnelaine
date : 20111024
encoder : Lavf60.16.100
comment : https://www.youtube.com/watch?v=RMM5N63d2DI
Duration: 00:01:24.83, start: 0.000000, bitrate: 269 kb/s
Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 480x360 [SAR 1:1 DAR 4:3], 133 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/22/2019.
vendor_id : [0][0][0][0]
Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/22/2019.
vendor_id : [0][0][0][0]
Stream #0:2[0x3](eng): Subtitle: mov_text (tx3g / 0x67337874), 0 kb/s (default)
Metadata:
handler_name : English
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x617ba5918f00] deprecated pixel format used, make sure you did set range correctly
[Parsed_thumbnail_1 @ 0x617ba5899e00] frame id #11 (pts_time=11.000000) selected from a set of 85 images
[vost#0:0/mjpeg @ 0x617ba5899640] No filtered frames for output stream, trying to initialize anyway.
Output #0, image2, to 'output_cover.jpg':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Numbers in the Teens They start with a 1!!!! song clipnabber com
artist : casnelaine
date : 20111024
comment : https://www.youtube.com/watch?v=RMM5N63d2DI
encoder : Lavf60.16.100
Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, progressive), 960x720 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 1 fps, 1 tbn (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/22/2019.
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
[out#0/image2 @ 0x617ba58a4800] video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[out#0/image2 @ 0x617ba58a4800] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
frame= 0 fps=0.0 q=0.0 Lsize=N/A time=N/A bitrate=N/A speed=N/A
Notice [out#0/image2 @ 0x617ba58a4800] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
that denotes no output.
I'll address the deprecation messages and make the frame detection less strict. I need to make some research on this first.
I'll address the deprecation messages and make the frame detection less strict. I need to make some research on this first.
@deldesir thanks for both above:
- Should IIAB Calibre-Web use a generic cover/thumbnail/poster instead, when FFmpeg fails completely — if this is indeed regularly blocking people from sharing their own/community videos?
A generic cover is already in use (https://github.com/iiab/calibre-web/blob/master/cps/static/generic_cover.jpg). Feel free to propose another one if necessary.
- If so, what's likely the best way to trap + act on FFmpeg errors like this?
This error can be handled. I figured out it was because this specific video doesn't have a scene change significant enough to match the scene detection filter select=gt(scene\,0.1)
.
- This is not the most urgent bug of all. Still: among offline schools that create their own videos — this bug could potentially be very damaging — if there's no clear workaround (automated workaround!?) allowing them to move forward.
I propose to add a failure detection to the subprocess and to rerun it without the scene detection filter when the initial run fails.
This error can be handled.
Great, please make it failsafe (fault-tolerant) most important:
Goal = Resilient + Reliable (NOT complexity!)
Deluxe features (e.g. the "most elegant" possible thumbnail, ETC) can always be added at a later date.
@EMG70 can you retry your 2-step process now, on a freshly installed or fully upgraded IIAB Calibre-Web?
yt-dlp https://youtu.be/RMM5N63d2DI
Related:
A new VM was created. SUDO IIAB-DIAGNOSTICS -https://dpaste.com/8KJAEH5KH
The previously failed video yt-dlp https://youtu.be/RMM5N63d2DI with exclamation marks its title can now be uploaded successfully,downloading also tested OK.
Short Notice from the maintainer
After 6 years of more or less intensive programming on Calibre-Web, I need a break. The last few months, maintaining Calibre-Web has felt more like work than a hobby. I felt pressured and teased by people to solve "their" problems and merge PRs for "their" Calibre-Web. I have turned off all notifications from Github/Discord and will now concentrate undisturbed on the development of “my” Calibre-Web over the next few weeks/months.
I will look into the issues and maybe also the PRs from time to time, but don't expect a quick response from me.
Please also have a look at our Contributing Guidelines
Describe the bug/problem
A clear and concise description of what the bug is. If you are asking for support, please check our Wiki if your question is already answered there.
To Reproduce
Steps to reproduce the behavior:
Go to ''https://www.youtube.com/watch?v=RMM5N63d2DI&list=PL2cADzwbKPk2JDfsxn6_-pDYfvCK9Fwfh&index=5"
Numbers in the Teens They start with a 1!!!! song clipnabber com RMM5N63d2DI.mp4![image](https://github.com/iiab/calibre-web/assets/52469992/cc4e53cc-756f-4405-941f-438083154ebb)
Click on "yt-dlp -S res,ext:mp4:m4a --recode mp4 https://www.youtube.com/watch?v=RMM5N63d2DI&list=PL2cADzwbKPk2JDfsxn6_-pDYfvCK9Fwfh&index=5"
Scroll down to '....'
See error
![Screenshot from 2024-07-06 04-33-30](https://github.com/iiab/calibre-web/assets/52469992/489b59b1-8ff9-4f53-86db-3f96da8daf5f)
[1] 14573 [2] 14574 root@ThinkCentre:/home/edwin# [youtube] Extracting URL: https://www.youtube.com/watch?v=RMM5N63d2DI [youtube] RMM5N63d2DI: Downloading webpage [youtube] RMM5N63d2DI: Downloading ios player API JSON [youtube] RMM5N63d2DI: Downloading android player API JSON [youtube] RMM5N63d2DI: Downloading m3u8 information [info] RMM5N63d2DI: Downloading 1 format(s): 135+140 [download] Numbers in the Teens They start with a 1!!!! song clipnabber com [RMM5N63d2DI].mp4 has already been downloaded [VideoConvertor] Not converting media file "Numbers in the Teens They start with a 1!!!! song clipnabber com [RMM5N63d2DI].mp4"; already is in target format mp4
Logfile https://dpaste.com/6ELYQTMWW Add content of calibre-web.log file or the relevant error, try to reproduce your problem with "debug" log-level to get more output.
Expected behavior Video's thumbnail is expected to upload successfully.![Screenshot from 2024-07-06 04-29-53](https://github.com/iiab/calibre-web/assets/52469992/de66c496-c3e5-44bf-aa02-ff8c3dc8fd6c)
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information): ubuntu@box:~$ hostnamectl Static hostname: box Icon name: computer-vm Chassis: vm 🖴 Machine ID: 79caaeb597524b9d96fd39c79a3cbb18 Boot ID: efc8827b49a847ec90822a9ffe062cd3 Virtualization: kvm Operating System: Ubuntu Oracular Oriole (development branch) Kernel: Linux 6.8.0-31-generic Architecture: x86-64 Hardware Vendor: QEMU Hardware Model: Standard PC Q35 + ICH9, 2009 Firmware Version: unknown Firmware Date: Wed 2022-02-02 Firmware Age: 2y 5month 2d
Additional context Add any other context about the problem here. [e.g. access via reverse proxy, database background sync, special database location]