export fails, running mlt-melt in a terminal on the mlt jobs works, meta: bugs closed without root causes being investigated

[ohsix]

when exporting, shotcut frequently fails to run the melt job, running them with mlt-melt succeeds

first-pass.mlt.txt second-pass.mlt.txt

this happens frequently, nearly every time

i must emphasize that a video editor that cannot export the results is not a useful program.

previous bugs have been closed with vague blame assigned to packagers with no reference to the root cause or any analysis of the problem. some include patches where the issue has not been differentiated from other fails with the same outcome:

Failed to load "%2Ftmp%2Ffirst-pass.mlt?multi:1" Failed to load "%2Ftmp%2Ffirst-pass.mlt?multi:1"

since running the mlt jobs manually with mlt-melt works, and there's no other information to begin to differentiate the problem based on log contents alone

strace shows that mlt-melt is showing shotcut the usage for having passed an invalid parameter

2236213 write(1, "Usage: mlt-melt [options] [producer [name=value] ]+\nOptions:\n -attach filter[:arg] [name=value] Attach a filter to the output\n -attach-cut filter[:arg] [name=value] Attach a filter to a cut\n -attach-track filter[:arg] [name=value] Attach a filter to a track\n -attach-clip filter[:arg] [name=value] Attach a filter to a producer\n -audio-track | -hide-video Add an audio-only track\n -blank frames Add blank silence to a track\n -consumer id[:arg] [name=value] Set the consumer (sink)\n -debug Set the logging level to debug\n -filter filter[:arg] [name=value] Add a filter to the current track\n -getc Get keyboard input using getc\n -group [name=value] Apply properties repeatedly\n -help Show this message\n -jack Enable JACK transport synchronization\n -join clips Join multiple clips into one cut\n -mix length Add a mix between the last two cuts\n -mixer transition Add a transition to the mix\n -null-track | -hide-track Add a hidden track\n -profile name Set the processing settings\n -progress Display progress along with position\n -query List all of the registered services\n -query \"consumers\" | \"consumer\"=id List consumers or show info about one\n -query \"filters\" | \"filter\"=id List filters or show info about one\n -query \"producers\" | \"producer\"=id List producers or show info about one\n -query \"transitions\" | \"transition\"=id List transitions, show info about one\n -query \"profiles\" | \"profile\"=id List profiles, show info about one\n -query \"presets\" | \"preset\"=id List presets, show info about one\n -query \"formats\" List audio/video formats\n -query \"audio_codecs\" List audio codecs\n -query \"video_codecs\" List video codecs\n -quiet Set the logging level to quiet\n -remove Remove the most recent cut\n -repeat times Repeat the last cut\n -repository path Set the directory of MLT modules\n -serialise [filename] Write the commands to a text file\n -silent Do not display position/transport\n -split relative-frame Split the last cut into two cuts\n -swap Rearrange the last two cuts\n -track Add a track\n -transition id[:arg] [name=value] Add a transition\n -verbose Set the logging level to verbose\n -timings Set the logging level to timings\n -version Show the version and copyright\n -video-track | -hide-audio Add a video-only track\nFor more help: https://www.mltframework.org/\n", 3212) = 3212 2229951 <... poll resumed>) = 1 ([{fd=92, revents=POLLIN}]) 2229951 ioctl(92, FIONREAD, [3212]) = 0 2229951 read(92, "Usage: mlt-melt [options] [producer [name=value] ]+\nOptions:\n -attach filter[:arg] [name=value] Attach a filter to the output\n -attach-cut filter[:arg] [name=value] Attach a filter to a cut\n -attach-track filter[:arg] [name=value] Attach a filter to a track\n -attach-clip filter[:arg] [name=value] Attach a filter to a producer\n -audio-track | -hide-video Add an audio-only track\n -blank frames Add blank silence to a track\n -consumer id[:arg] [name=value] Set the consumer (sink)\n -debug Set the logging level to debug\n -filter filter[:arg] [name=value] Add a filter to the current track\n -getc Get keyboard input using getc\n -group [name=value] Apply properties repeatedly\n -help Show this message\n -jack Enable JACK transport synchronization\n -join clips Join multiple clips into one cut\n -mix length Add a mix between the last two cuts\n -mixer transition Add a transition to the mix\n -null-track | -hide-track Add a hidden track\n -profile name Set the processing settings\n -progress Display progress along with position\n -query List all of the registered services\n -query \"consumers\" | \"consumer\"=id List consumers or show info about one\n -query \"filters\" | \"filter\"=id List filters or show info about one\n -query \"producers\" | \"producer\"=id List producers or show info about one\n -query \"transitions\" | \"transition\"=id List transitions, show info about one\n -query \"profiles\" | \"profile\"=id List profiles, show info about one\n -query \"presets\" | \"preset\"=id List presets, show info about one\n -query \"formats\" List audio/video formats\n -query \"audio_codecs\" List audio codecs\n -query \"video_codecs\" List video codecs\n -quiet Set the logging level to quiet\n -remove Remove the most recent cut\n -repeat times Repeat the last cut\n -repository path Set the directory of MLT modules\n -serialise [filename] Write the commands to a text file\n -silent Do not display position/transport\n -split relative-frame Split the last cut into two cuts\n -swap Rearrange the last two cuts\n -track Add a track\n -transition id[:arg] [name=value] Add a transition\n -verbose Set the logging level to verbose\n -timings Set the logging level to timings\n -version Show the version and copyright\n -video-track | -hide-audio Add a video-only track\nFor more help: https://www.mltframework.org/\n", 3212) = 3212

the command line in use:

execve("/usr/bin/nice", ["/usr/bin/nice", "-n", "3", "/usr/bin/mlt-melt", "-verbose", "-progress2", "-abort", "%2Ftmp%2Fshotcut-KBmCjP.mlt?multi:1"]

when ran in a shell:

[ohsix@pro ~]☁️ "/usr/bin/nice" "-n" "3" "/usr/bin/mlt-melt" "-verbose" "-progress2" "-abort" "%2Ftmp%2Ffirst-pass.mlt.txt?multi:1" plugin_mgr_get_object_file_plugins: error finding ladspa_descriptor symbol in object file '/usr/lib64/ladspa/calf.so': /usr/lib64/ladspa/calf.so: undefined symbol: ladspa_descriptor plugin_mgr_get_object_file_plugins: error opening shared object file '/usr/lib64/ladspa/ladspa-rubberband.cat': /usr/lib64/ladspa/ladspa-rubberband.cat: invalid ELF header Failed to load "%2Ftmp%2Ffirst-pass.mlt.txt?multi:1" Failed to load "%2Ftmp%2Ffirst-pass.mlt.txt?multi:1" Usage: mlt-melt [options] [producer [name=value]* ]+ Options:

with ?multu=1 removed

[ohsix@pro ~]☁️ "/usr/bin/nice" "-n" "3" "/usr/bin/mlt-melt" "-verbose" "-progress2" "-abort" "%2Ftmp%2Ffirst-pass.mlt.txt" plugin_mgr_get_object_file_plugins: error finding ladspa_descriptor symbol in object file '/usr/lib64/ladspa/calf.so': /usr/lib64/ladspa/calf.so: undefined symbol: ladspa_descriptor plugin_mgr_get_object_file_plugins: error opening shared object file '/usr/lib64/ladspa/ladspa-rubberband.cat': /usr/lib64/ladspa/ladspa-rubberband.cat: invalid ELF header [swscaler @ 0x5596f16c12c0] Warning: data is not aligned! This can lead to a speed loss [consumer sdl2] Audio Opened: driver=pulseaudio channels=2 frequency=48000 Current Frame: 10, percentage: 0 ...

a patch for one of the improperly closed bugs shows that xml: was added as a fix https://github.com/mltframework/shotcut/issues/1079

in this case, mlt-melt still "fails", but not from an invalid parameter per-se

[ohsix@pro ~]☁️ "/usr/bin/nice" "-n" "3" "/usr/bin/mlt-melt" "-verbose" "-progress2" "-abort" "xml:%2Ftmp%2Ffirst-pass.mlt.txt?multi:1" plugin_mgr_get_object_file_plugins: error finding ladspa_descriptor symbol in object file '/usr/lib64/ladspa/calf.so': /usr/lib64/ladspa/calf.so: undefined symbol: ladspa_descriptor plugin_mgr_get_object_file_plugins: error opening shared object file '/usr/lib64/ladspa/ladspa-rubberband.cat': /usr/lib64/ladspa/ladspa-rubberband.cat: invalid ELF header [libx264 @ 0x7fdcb82023c0] using SAR=1/1 [libx264 @ 0x7fdcb82023c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX Current Frame: 11, percentage: 57 [libx264 @ 0x7fdcb82023c0] profile Main, level 3.0, 4:2:0, 8-bit [libx264 @ 0x7fdcb82023c0] 264 - core 161 r3049 55d517b - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0 me=dia subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=149 keyint_min=14 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Current Frame: 18, percentage: 94 [mp4 @ 0x7fdcb8200f40] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.

[mp4 @ 0x7fdcb8200f40] Starting second pass: moving the moov atom to the beginning of the file [AVIOContext @ 0x7fdcb9eb0f00] Statistics: 0 bytes read, 0 seeks [libx264 @ 0x7fdcb82023c0] frame I:1 Avg QP: 6.00 size: 114 [libx264 @ 0x7fdcb82023c0] frame P:5 Avg QP: 3.60 size: 25 [libx264 @ 0x7fdcb82023c0] frame B:13 Avg QP: 7.54 size: 21 [libx264 @ 0x7fdcb82023c0] consecutive B-frames: 5.3% 10.5% 0.0% 84.2% [libx264 @ 0x7fdcb82023c0] mb I I16..4: 99.9% 0.0% 0.1% [libx264 @ 0x7fdcb82023c0] mb P I16..4: 0.0% 0.0% 0.0% P16..4: 0.0% 0.0% 0.0% 0.0% 0.0% skip:100.0% [libx264 @ 0x7fdcb82023c0] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% [libx264 @ 0x7fdcb82023c0] final ratefactor: -6.33 [libx264 @ 0x7fdcb82023c0] direct mvs spatial:30.8% temporal:69.2% [libx264 @ 0x7fdcb82023c0] coded y,uvDC,uvAC intra: 0.0% 0.0% 0.0% inter: 0.0% 0.0% 0.0% [libx264 @ 0x7fdcb82023c0] i16 v,h,dc,p: 97% 0% 3% 0% [libx264 @ 0x7fdcb82023c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0% [libx264 @ 0x7fdcb82023c0] i8c dc,h,v,p: 100% 0% 0% 0% [libx264 @ 0x7fdcb82023c0] Weighted P-Frames: Y:0.0% UV:0.0% [libx264 @ 0x7fdcb82023c0] kb/s:6.40 [AVIOContext @ 0x7fdcb9e35280] Statistics: 4 seeks, 4 writeouts

the patch that introduced the multi consumer stuff that didn't really work was https://github.com/mltframework/shotcut/commit/7a25c2485afd25006e4996e8b0208dccb6e67851

this was written in anger as the failures were seemingly random and obvious and i'd "just have to wait a bit for this obvious thing to be fixed", that did not happen

I suggest not eating the output of mlt in the log, as all of this would be directly visible in a failed job if it were not hidden.

probably a good idea to hide the meaningless ladspa errors, nobody cars if melt tells them there's a object in the ladspa plugin path without a ladspa_descriptor https://github.com/mltframework/mlt/blob/master/src/modules/jackrack/plugin_mgr.c#L97

josh

[ddennedy]

That is not a release or build made by us.

[ohsix]

wanna hear something funny? it wasn't even a fedora package until very recently. anyways, i'm really embarrassed and think about this bug regularly.

the bugs i've had since were definitely not "yours", and aside from crashes none of them have held me up like that again

shotcut is important to me and i'm glad it exists. thank you.