search

toolforge / video2commons

Transfer video and audio from external sites to Commons.
https://video2commons.toolforge.org/
GNU General Public License v3.0
43 stars 24 forks source link

Exitcode: 153 #210

Open Yannf opened 3 months ago

Yannf commented 3 months ago

I got An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b344e90a1e97be9a/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/b344e90a1e97be9a/dl.mp4.vp9.webm.log /srv/v2c/output/b344e90a1e97be9a/dl.mp4.vp9.webm\\nExitcode: 153')" while trying to upload https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov

Yannf commented 3 months ago

An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/11218d7b953b1eb7/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/11218d7b953b1eb7/dl.mp4.vp9.webm.log /srv/v2c/output/11218d7b953b1eb7/dl.mp4.vp9.webm\\nExitcode: 153')" for https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov

don-vip commented 3 months ago

Error happened on encoding01

[2024-08-03 11:21:33,709: WARNING/ForkPoolWorker-3] 38: Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/f44329abf55dfcac/dl.mkv -max_muxing_queue_size 4096 -vcodec copy -ss 0 -b:a 128000 -ar 48000 -acodec libopus /srv/v2c/output/f44329abf55dfcac/dl.mkv.vp9.webm
[2024-08-03 11:21:36,208: WARNING/ForkPoolWorker-3] -1: Cleaning up...
[2024-08-03 11:21:37,334: ERROR/ForkPoolWorker-3] Task video2commons.backend.worker.main[7c8e92c9-9fca-4934-a13b-fad97b40471f] raised unexpected: UnpickleableExceptionWrapper('video2commons.exceptions', 'TaskError', ('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/f44329abf55dfcac/dl.mkv -max_muxing_queue_size 4096 -vcodec copy -ss 0 -b:a 128000 -ar 48000 -acodec libopus /srv/v2c/output/f44329abf55dfcac/dl.mkv.vp9.webm\nExitcode: 153',), "TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/f44329abf55dfcac/dl.mkv -max_muxing_queue_size 4096 -vcodec copy -ss 0 -b:a 128000 -ar 48000 -acodec libopus /srv/v2c/output/f44329abf55dfcac/dl.mkv.vp9.webm\\nExitcode: 153')")
Traceback (most recent call last):
  File "/srv/v2c/venv/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/venv/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/video2commons/backend/worker.py", line 123, in main
    file = encode.encode(file, convertkey, statuscallback, errorcallback)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/video2commons/backend/encode/__init__.py", line 51, in encode
    return target if job.run() else None
                     ^^^^^^^^^
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 181, in run
    self.set_error(status, transcode_key)
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 115, in set_error
    self.errorcallback(error)
  File "/srv/v2c/video2commons/backend/worker.py", line 107, in errorcallback
    raise TaskError(text)
celery.utils.serialization.UnpickleableExceptionWrapper: TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/f44329abf55dfcac/dl.mkv -max_muxing_queue_size 4096 -vcodec copy -ss 0 -b:a 128000 -ar 48000 -acodec libopus /srv/v2c/output/f44329abf55dfcac/dl.mkv.vp9.webm\nExitcode: 153')

image

don-vip commented 3 months ago

I have no idea what happened. I cleaned up a little bit the filesystem but I'm not sure it's because of a disk space issue. Nothing useful in celery logs nor syslog...

chicocvenancio commented 3 months ago

Its a proper Ffmpeg error, we'd have to debug the command for this specific video file.

Jeff-G-commons commented 2 months ago

Here's a recent error: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/70706c79a23d4312/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm.log /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm\\nExitcode: 153')"

don-vip commented 2 months ago

Error happened on encoding05

[2024-08-18 23:33:31,664: WARNING/ForkPoolWorker-3] 94: Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/70706c79a23d4312/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm.log /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm
[2024-08-18 23:34:06,503: WARNING/ForkPoolWorker-3] -1: Cleaning up...
[2024-08-18 23:34:07,102: ERROR/ForkPoolWorker-3] Task video2commons.backend.worker.main[6a8810fa-3d5d-4d12-8574-fd289bd775fc] raised unexpected: UnpickleableExceptionWrapper('video2commons.exceptions', 'TaskError', ('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/70706c79a23d4312/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm.log /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm\nExitcode: 153',), "TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/70706c79a23d4312/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm.log /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm\\nExitcode: 153')")
Traceback (most recent call last):
  File "/srv/v2c/venv/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/venv/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/video2commons/backend/worker.py", line 123, in main
    file = encode.encode(file, convertkey, statuscallback, errorcallback)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/v2c/video2commons/backend/encode/__init__.py", line 51, in encode
    return target if job.run() else None
                     ^^^^^^^^^
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 181, in run
    self.set_error(status, transcode_key)
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 115, in set_error
    self.errorcallback(error)
  File "/srv/v2c/video2commons/backend/worker.py", line 107, in errorcallback
    raise TaskError(text)
celery.utils.serialization.UnpickleableExceptionWrapper: TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/70706c79a23d4312/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm.log /srv/v2c/output/70706c79a23d4312/dl.mp4.an.vp9.webm\nExitcode: 153')
Jeff-G-commons commented 2 months ago

Its a proper Ffmpeg error, we'd have to debug the command for this specific video file.

Exactly where may we find documentation of that exitcode? https://ffmpeg.org/documentation.html is of no help to me in my search.

don-vip commented 2 months ago

Wow I see we're using a very old version of ffmpeg, first thing would be to upgrade it:

$ /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -version
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Gentoo 10.2.0-r5 p6)
don-vip commented 2 months ago

I'm trying to update the Gentoo Prefix, it hasn't been done since 2021 :'(

don-vip commented 2 months ago

OK it's way too complicated to update with so many skipped versions, and I managed to completely screw the gentoo installation. I'm reinstalling the system, :fire: :fire: :fire: expect all new jobs to fail in the coming hours :fire: :fire: :fire: until I complete this task.

don-vip commented 2 months ago

I've stopped the frontend so that new tasks cannot be created until I repair the backend.

don-vip commented 2 months ago

Gentoo has been reinstalled and ffmpeg updated. I have restarted the service but not tested anything yet. Can some people please tell me if it works?

don-vip commented 2 months ago

New version is built as follows:

$ /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -version
ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13 (Gentoo 13.3.1_p20240614 p17)
configuration: --prefix=/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr --libdir=/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/lib64 --shlibdir=/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/lib64 --docdir=/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/share/doc/ffmpeg-6.1.1-r8/html --mandir=/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --nm=x86_64-pc-linux-gnu-nm --strip=x86_64-pc-linux-gnu-strip --ranlib=x86_64-pc-linux-gnu-ranlib --pkg-config=x86_64-pc-linux-gnu-pkg-config --optflags='-O2 -pipe -O2 -pipe' --disable-static --disable-libaribcaption --enable-avfilter --disable-stripping --disable-optimizations --disable-libcelt --enable-nonfree --enable-version3 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=jack --enable-version3 --enable-version3 --enable-version3 --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --enable-gcrypt --enable-gnutls --enable-gmp --enable-gpl --disable-hardcoded-tables --enable-iconv --enable-libxml2 --enable-lzma --enable-network --disable-opencl --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --enable-vaapi --enable-vdpau --disable-vulkan --disable-xlib --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --enable-openal --disable-opengl --disable-libv4l2 --disable-libpulse --disable-libdrm --disable-libjack --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libcodec2 --enable-libdav1d --enable-libfdk-aac --enable-libopenjpeg --enable-libjxl --enable-libbluray --enable-libgme --enable-libgsm --enable-libaribb24 --disable-libmodplug --enable-libopus --disable-libvpl --enable-libilbc --enable-librtmp --disable-libssh --disable-libspeex --enable-libsrt --enable-librsvg --disable-ffnvcodec --enable-libvorbis --enable-libvpx --enable-libzvbi --disable-appkit --enable-libbs2b --enable-chromaprint --disable-cuda-llvm --disable-libflite --enable-fontconfig --disable-frei0r --enable-libfribidi --disable-libglslang --disable-ladspa --enable-lcms2 --enable-libass --disable-libplacebo --disable-libtesseract --enable-lv2 --enable-librubberband --disable-libshaderc --enable-libfreetype --enable-libharfbuzz --disable-libvidstab --disable-libvmaf --disable-libzmq --enable-libzimg --enable-libsoxr --enable-pthreads --disable-amf --enable-libvo-amrwbenc --disable-libkvazaar --enable-libaom --enable-libmp3lame --enable-libopenh264 --disable-librav1e --disable-libsnappy --enable-libsvtav1 --enable-libtheora --disable-libtwolame --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --disable-gnutls --enable-version3 --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-dotprod --disable-i8mm --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-vsx --disable-power8 --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --enable-pic --disable-doc --disable-htmlpages --enable-manpages
libavutil      58. 29.100 / 58. 29.100
libavcodec     60. 31.102 / 60. 31.102
libavformat    60. 16.100 / 60. 16.100
libavdevice    60.  3.100 / 60.  3.100
libavfilter     9. 12.100 /  9. 12.100
libswscale      7.  5.100 /  7.  5.100
libswresample   4. 12.100 /  4. 12.100
libpostproc    57.  3.100 / 57.  3.100
don-vip commented 2 months ago

@Yannf @Jeff-G-commons I have managed to upload several videos so it's working again. Can you please try again the videos that caused the Error 153, to see if it's solved with the ffmepg update?

Yannf commented 2 months ago

OK, started again. Your task is pending...

Jeff-G-commons commented 2 months ago

OK, I restarted one (Dixiana 1930 (colorized, 4k, 60FPS) 2), two more are Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler (94%) and Storm Over Asia (1928, 1080p, silent) (89%) by Vsevolod Pudovkin in pass 2 (percentages increased since you asked, so they are still running and I am hesitant to abort them). I have no percentages to relay when in pass 1, just a horizontal barber pole. Also, the info in the column where the errors show is difficult to parse at a glance, and the columns are not labeled.

Jeff-G-commons commented 2 months ago

Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b6f1fff950a5fae9/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm.log /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Similarly, Storm Over Asia (1928, 1080p, silent) by Vsevolod Pudovkin had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/ccb74212f21d3a91/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm.log /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b6f1fff950a5fae9/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm.log /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/8ecad562d8da07f2/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm.log /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Similarly, Storm Over Asia (1928, 1080p, silent) by Vsevolod Pudovkin had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/ccb74212f21d3a91/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm.log /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a61d2fa98b38123c/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm.log /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm\nExitcode: 153')"

BahYaje commented 2 months ago

UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/5f1d66720427aa6c/dl.mkv -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -ss 0 -an -pass 1 -passlogfile /srv/v2c/output/5f1d66720427aa6c/dl.mkv.vp9.webm.log /dev/null\nExitcode: 8')"

don-vip commented 2 months ago

... l\nExitcode: 8')"

@BahYaje this ticket is only about the ExitCode 153, can you please create a new ticket for your issue with the link to the video files you're trying to upload? thanks.

Yannf commented 2 months ago

Volshebnoye Zerno - The Magic Seed (1942) by Fyodor Filippov

An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/1f6495e96e57e8e8/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/1f6495e96e57e8e8/dl.mp4.vp9.webm.log /srv/v2c/output/1f6495e96e57e8e8/dl.mp4.vp9.webm\\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b6f1fff950a5fae9/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm.log /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/8ecad562d8da07f2/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm.log /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm.log /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Similarly, Storm Over Asia (1928, 1080p, silent) by Vsevolod Pudovkin had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/ccb74212f21d3a91/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm.log /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a61d2fa98b38123c/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm.log /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a461584cb01f01b0/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm.log /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Dixiana 1930 (colorized, 4k, 60FPS) 2 had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/49fd5ad65ee6abef/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm.log /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Fragment of an Empire (1929, 1080p, silent) by Fridrikh Ermler had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/b6f1fff950a5fae9/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm.log /srv/v2c/output/b6f1fff950a5fae9/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/8ecad562d8da07f2/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm.log /srv/v2c/output/8ecad562d8da07f2/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm.log /srv/v2c/output/9385eedfaa0b5cf6/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4.an.vp9.webm.log /srv/v2c/output/3c85cf9ab7375c9e/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Similarly, Storm Over Asia (1928, 1080p, silent) by Vsevolod Pudovkin had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/ccb74212f21d3a91/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm.log /srv/v2c/output/ccb74212f21d3a91/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a61d2fa98b38123c/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm.log /srv/v2c/output/a61d2fa98b38123c/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a461584cb01f01b0/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm.log /srv/v2c/output/a461584cb01f01b0/dl.mp4.an.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/08c147bc201c1eed/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -an -pass 2 -passlogfile /srv/v2c/output/08c147bc201c1eed/dl.mp4.an.vp9.webm.log /srv/v2c/output/08c147bc201c1eed/dl.mp4.an.vp9.webm\nExitcode: 153')"

Jeff-G-commons commented 2 months ago

Dixiana 1930 (colorized, 4k, 60FPS) 2 had the following exit, I restarted: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/49fd5ad65ee6abef/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm.log /srv/v2c/output/49fd5ad65ee6abef/dl.mp4.vp9.webm\nExitcode: 153')"

Latest error follows: An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/782c3b089253acde/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/782c3b089253acde/dl.mp4.vp9.webm.log /srv/v2c/output/782c3b089253acde/dl.mp4.vp9.webm\nExitcode: 153')"

don-vip commented 2 months ago

@Jeff-G-commons I have enough reported errors, thank you. No need to report every one of them.

Jeff-G-commons commented 2 months ago

@Jeff-G-commons I have enough reported errors, thank you. No need to report every one of them.

You're welcome.

don-vip commented 2 months ago

I managed today to convert manually the first reported file (https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov) on encoding06 instance:

$ python encode.py /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4 vp9.webm
Encoding to codec: vp9
Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -an -pass 1 -passlogfile /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4.vp9.webm.log /dev/null
Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -b:a 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4.vp9.webm.log /home/don-vip/Volshebnoye.Zerno.1942.1080p.BluRay.mp4.vp9.webm

$ ls -l Volshebnoye.Zerno.1942.1080p.BluRay.mp4*
-rw-r--r-- 1 don-vip wikidev 1541959023 Jun  1 07:43 Volshebnoye.Zerno.1942.1080p.BluRay.mp4
-rw-r--r-- 1 don-vip wikidev 5368709120 Aug 25 00:00 Volshebnoye.Zerno.1942.1080p.BluRay.mp4.vp9.webm

It took exactly 12 hours. Consumed only around 700Mb of memory during the process. And produced a file of 5 GB:

image

image

image

(the CPU spike is just another job that someone launched. The disk space event was me freeing some old unused files).

I don't think it's an ffmpeg error, as it succeeded with this example. Error 153 might be the process killed by a SIGINT.

video2commons launches jobs with some ulimit parameters that might explain why a SIGINT is sent by the system:

        cmd = 'ulimit -f ' + escape_shellarg(background_size_limit) + ';' + \
            'ulimit -v ' + escape_shellarg(background_memory_limit) + ';' + \
            'nice -n ' + escape_shellarg(background_priority) + ' ' + \
            'timeout ' + escape_shellarg(background_time_limit) + ' ' + \
            cmd + \
            ' 2>&1'

The values are defined in globals.py:

# The priority to be used with the nice transcode commands.
background_priority = 19
# The total amout of time a transcoding shell command can take:
background_time_limit = 3600 * 24 * 2  # 2 days
# Maximum amount of virtual memory available to transcoding processes in KB
# 2GB avconv
background_memory_limit = 8 * 1024 * 1024  # 8GB
# Maximum file size transcoding processes can create, in KB
background_size_limit = 10 * 1024 * 1024  # 10GB

But in this case we are way below the limits... So I still don't understand what's going on.

Jeff-G-commons commented 2 months ago

@don-vip Thanks. Were you trying to upload that film with or without sound? @Yannf is the sound free enough for Commons? All of my current failed uploads listed above are without sound because the sound is not free enough for Commons (except Dixiana).

Jeff-G-commons commented 2 months ago

@Yannf I am happy to report that I was able to upload https://archive.org/details/volshebnoye-zerno-1942-by-fyodor-filippov to https://commons.wikimedia.org/wiki/File:Volshebnoye_Zerno_-_The_Magic_Seed_%281942%2C_silent%29.webm . Please attend to the licensing.

Jeff-G-commons commented 1 month ago

I now have three instances of 'An exception occurred: ConnectionError: b"Error while reading from video-redis-bookworm.video.eqiad1.wikimedia.cloud:6379 : (104, 'Connection reset by peer')"' with the same files. Should that be a separate issue?

Jeff-G-commons commented 1 month ago

Grafana shows that there was a dip in Total RAM to 31.3 GiB 2024-09-17 11:20.

don-vip commented 1 month ago

Please try again, the outage lasted only for 10 minutes, I think it's simply a maintenance operation from WMCS team.

Jeff-G-commons commented 3 weeks ago

Over a month later, are you any closer to understanding what's going on?