Issue using multiple -attach watermark:filename

[jacksonvm]

Hi, I have been experimenting in overlaying multiple layers of Graphics files rendered in AVI with alpha in mlt, by creating the XML. I'm successfully able to overlay two layers of AVI /QTRLE mov/vp6a FLV. When i add a third layer, the Melted starts creating multiple threads, in a back to back file playback scenario and finally crashes.

Thanks,

Jackson

[ddennedy]

Please provide a better description. How about a melt command line that reproduces the problem? Also provide a version and OS.

[jacksonvm]

Hi Dan

Below provided is an XML used for overlaying GFX using composite transition. The OS used is Cent OS 6.5 and the version of melt is 6.5.0. I even tried replacing composite transition with frei0r.cairoblend (In this case the melted becomes non responsive while appending a new clip).

ps -Huh -p PID of melted is used to monitor the threads.

NOTE - Im using CBRTS output with melted.

<mlt>
   <playlist id="background">
    <entry producer="black" in="00:00:00.000" out="00:00:19.960"/>
  </playlist>
  <producer id="producer0" in="00:00:00.000" out="00:00:19.960">
    <property name="resource">SC038822.mxf</property>
    </producer>
  <playlist id="playlist0">
    <entry producer="producer0" in="00:00:00.000" out="00:00:19.960"/>
  </playlist>
  <producer id="producer1" title="Anonymous Submission" in="00:00:00.000" out="00:00:14.960">
     <property name="resource">g1.avi</property>
     </producer>
  <playlist id="playlist1">
       <entry producer="producer1" in="00:00:00.000" out="00:00:14.960"/>
  </playlist>
  <producer id="producer2" title="Anonymous Submission" in="00:00:00.000" out="00:00:10.000">
    <property name="resource">g2.avi</property>
     </producer>
  <producer id="producer3" title="Anonymous Submission" in="00:00:00.000" out="00:00:10.000">
     <property name="resource">g2.avi</property>
     </producer>
  <playlist id="playlist2">
   <entry producer="producer2" in="00:00:00.000" out="00:00:10.000"/>
   <entry producer="producer3" in="00:00:00.000" out="00:00:10.000"/>
  </playlist>
  <producer id="producer4" title="Anonymous Submission" in="00:00:00.000" out="00:00:09.960">
     <property name="resource">g3.avi</property>
     </producer>
  <producer id="producer5" title="Anonymous Submission" in="00:00:00.000" out="00:00:09.960">
    <property name="resource">g3.avi</property>
    </producer>
  <playlist id="playlist3">
    <entry producer="producer4" in="00:00:00.000" out="00:00:09.960"/>
    <entry producer="producer5" in="00:00:00.000" out="00:00:09.960"/>
  </playlist>
  <tractor id="tractor0" title="Shotcut version 15.11.03" global_feed="1" in="00:00:00.000" out="00:00:19.960">
    <property name="shotcut">1</property>
    <track producer="background"/>
    <track producer="playlist0"/>
    <track producer="playlist1"/>
    <track producer="playlist2"/>
    <track producer="playlist3"/>
    <transition id="transition0">
      <property name="a_track">0</property>
      <property name="b_track">1</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">mix</property>
      <property name="always_active">1</property>
      <property name="combine">1</property>
    </transition>
      <transition id="transition2">
      <property name="a_track">0</property>
      <property name="b_track">2</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">mix</property>
      <property name="always_active">1</property>
      <property name="combine">1</property>
    </transition>
    <transition id="transition3">
      <property name="a_track">1</property>
      <property name="b_track">2</property>
      <property name="version">0.9</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">composite</property>
      <property name="disable">0</property>
    </transition>
    <transition id="transition4">
      <property name="a_track">0</property>
      <property name="b_track">3</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">mix</property>
      <property name="always_active">1</property>
      <property name="combine">1</property>
    </transition>
    <transition id="transition5">
      <property name="a_track">1</property>
      <property name="b_track">3</property>
      <property name="version">0.9</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">composite</property>
      <property name="disable">0</property>
    </transition>
    <transition id="transition6">
      <property name="a_track">0</property>
      <property name="b_track">4</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">mix</property>
      <property name="always_active">1</property>
      <property name="combine">1</property>
    </transition>
    <transition id="transition7">
      <property name="a_track">1</property>
      <property name="b_track">4</property>
      <property name="version">0.9</property>
      <property name="mlt_type">transition</property>
      <property name="mlt_service">composite</property>
      <property name="disable">0</property>
    </transition>
  </tractor>
</mlt> "

Regards,

Jackson

[ddennedy]

In order to better recreate the scenario, I want to use or make something similar to your AVI files because they appear to have an alpha channel to support your use case. Please share some details about them such as ffprobe output. Or, you can upload one to https://shotcut.org/notes/support-upload/ and that will make my task to reproduce this more convenient.

[jacksonvm]

Hi Dan,

Thank you for looking into it. The AVI files has been uploaded in the below link for testing.

http://42.99.180.212/share/index.php/s/Rg2v5g05UluSuEn password -12345

Kindly let me know incase you need any more info to debug the issue.

Regards,

Jackson

[ddennedy]

I got your files and got your MLT XML to work. Your AVI files are using the lagarith codec with bgra pixel format. I am not experiencing a problem when using melt with the sdl consumer. I really doubt melted is a problem. The cbrts consumer is no longer working for me since FFmpeg v3.x the last time I tried. Are you successfully using the cbrts consumer outside of this problem? If so, what version of FFmpeg are you using with MLT?

[jacksonvm]

Hi Dan,

Yes Im able to use the CBRTS consumer successfully with two layers of watermark/Composition without any issues.Infact the CBRTS output with two layer of watermark/Composition from MELTED is been locked and decoded by a professional decoder for more than 48 hrs. The issue only starts when we try to add the third layer. The threads of the MELTED PROCESS starts increasing when playing clips back to back and crashes after a while.

The FFmpeg version details are as mentioned below

ffmpeg version N-79160-gb73c271 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4) configuration: --prefix=/usr/local/AV --extra-cflags=-I/usr/local/AV/include --extra-ldflags=-L//usr/local/AV/lib --extra-cflags=-I/home/sw_dump/BLACKMAGIC/Blackmagic_Decklink_SDK_10.0/Linux/include --enable-gpl --enable-shared --enable-pthreads --enable-libmp3lame --enable-swscale --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-version3 --enable-x11grab --enable-nonfree --enable-pic --enable-avresample --enable-libfreetype --enable-filter=drawtext --enable-ffplay --enable-libfdk-aac libavutil 55. 19.100 / 55. 19.100 libavcodec 57. 32.100 / 57. 32.100 libavformat 57. 29.101 / 57. 29.101 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 40.102 / 6. 40.102 libavresample 3. 0. 0 / 3. 0. 0 libswscale 4. 0.100 / 4. 0.100 libswresample 2. 0.101 / 2. 0.101 libpostproc 54. 0.100 / 54. 0.100

CBRTS Conf

uadd cbrts: uset u0 consumer.muxrate=3000008 uset u0 consumer.vb=1500000 uset u0 consumer.minrate=1500000 uset u0 consumer.maxrate=1500000 uset u0 consumer.bufsize=1835008 uset u0 consumer.g=15 uset u0 consumer.bf=2 uset u0 consumer.vcodec=libx264 uset u0 consumer.channels.0=2 uset u0 consumer.channels.1=1 uset u0 consumer.acodec=libfdk_aac uset u0 consumer.ab=160000 uset u0 consumer.ar=48000 uset u0 consumer.threads=4 uset u0 consumer.udp.address=225.7.2.3 uset u0 consumer.udp.port=9005 uset u0 consumer.udp.reuse=1 uset u0 consumer.udp.nb_tsp=7 uset u0 consumer.udp.ttl=10 uset u0 consumer.udp.rtprio=99 uset u0 consumer.udp.rtp=0

Hope this information gives you a clear picture of the issue.

Regards,

Jackson

[ddennedy]

I was finally able to get cbrts working with recent FFmpeg, but I am not using rtprio=99. However, I have not reproduced the problem.

[jacksonvm]

Hi Dan,

Can you please provide the MELT syntax you are using to create the XML. Also which version of LINUX are you using? As i have tried the same settings on a Ubuntu 16.04 and CentOS 6.5. Still I'm facing the same issue when i run clips.

Regards,

Jackson

[ddennedy]

I edited your MLT XML to refer to a different video clip than the MXF you use. That is the only difference. My video clip is a PAL SD MPEG-2 program stream. I am using Ubuntu 16.04.

The only reason I can see that it may be creating additional threads with each clip is because libavcodec is allocating multiple decoding threads (it can do that). By default, mlt_playlist does not close each producer after finishing playing it because MLT is designed for random access (seeking). However, due to a producer caching mechanism, the libavformat and libavcodec data objects, including threads, should eventually be destroyed due to more new producers going into the cache and flushing out the least recently used producer. In other words, if closely monitoring thread counts, it is normal for thread counts go up for the first 5 clips and then remain fairly steady at a certain level (assuming cache level defaults). MLT and its modules do not normally dynamically create and destroy threads except for a new feature in v6.4.0: multi-threaded slice-based image processing. The composite transition does take advantage of this but not by default - you have to set a property to opt into it. It is possible (but unlikely) that some heap corruption is causing that to trigger. You can try to use MLT v6.2.0, which does not include this feature to see if it makes a difference.

[jacksonvm]

Hi Dan,

I compiled the MLT 6.2.0, with FFmpeg 2.3.6 and vpx 1.3.0 on fresh Ubuntu 14.04 machine, but still the threads were getting increased. Then I compiled using your build script on a fresh machine and it seems like the the thread issue is resolved but cbrts is not smooth.

Can you please let me know the ffmpeg version you are using with the CBRTS Consumer on MLT 6.5.0?

Regards,

Jackson

[ddennedy]

I do not really use the cbrts consumer such as in a broadcast fashion. I only compile and test it, and I no longer have a recent combination of versions that is certified to work. It was about Sept/Oct 2015 when I did run cbrts through an analyzer. FFmpeg v2.6.3 was made a few months earlier to that and probably works.

You can see here, there has been very little changes to cbrts consumer. In the top of the build script, you can set a specific FFmpeg version using a tag such as "n2.6.3". If you use that, then I suggest you remove the "src" directory within your melted build directory to make a clean build. FFMPEG_HEAD=0 FFMPEG_REVISION=n2.6.3

[ddennedy]

Your output might not be smooth because you are giving it too much work to do: compositing 4 layers at full resolution plus H.264 and AAC encoding all in real time. If you have at least 8 logical CPUs, you can try setting real_time on the consumer to -2 or -3: uset u0 consumer.real_time=-3

[jacksonvm]

Hi Dan,

Thank you very much for your support uset u0 consumer.real_time=-3 has worked well for me. I have a working version of MLT 6.5.0 on Centos 6.5, Centos 7 . One issue I'm facing right now is melt crashes after some time. While debugging it was noted when checking with lsof the files are not released after playback. This only occurs when I try to apply more than two layers of composition, I have tried using autoclose=1 to close the playlist as you have mentioned in one of the stackoverflow comments.

Thanks and Regards,

Jackson

[jacksonvm]

Hi Dan,

I managed to get a workaround by specifying an Offset in the GFX watermark composition.

Also note that I have been working on the UDP producer (avformat-novalidate) as mentioned in one of the Q&A in your forums.

This seems to work fine with MELT 0.9.9 and doesnt work with any of the other versions.

<?xml version="1.0" encoding="utf-8"?>
<mlt LC_NUMERIC="en_IN" version="0.9.9" root="/home/test/Videos" title="127.0.0.1:1234?fifo_size=1000000&amp;overrun_nonfatal=1">
  <profile description="automatic" width="720" height="576" progressive="0" sample_aspect_num="16" sample_aspect_den="15" display_aspect_num="768" display_aspect_den="576" frame_rate_num="25" frame_rate_den="1" colorspace="601"/>
  <producer id="producer0" in="0" out="2160000">
    <property name="mlt_type">producer</property>
    <property name="length">2160000</property>
    <property name="eof">loop</property>
    <property name="resource">udp://127.0.0.1:1234?fifo_size=1000000&amp;overrun_nonfatal=1</property>
    <property name="meta.media.nb_streams">2</property>
    <property name="meta.media.0.stream.type">video</property>
    <property name="meta.media.0.stream.frame_rate">25</property>
    <property name="meta.media.0.stream.sample_aspect_ratio">0</property>
    <property name="meta.media.0.codec.width">720</property>
    <property name="meta.media.0.codec.height">576</property>
    <property name="meta.media.0.codec.rotate">0</property>
    <property name="meta.media.0.codec.frame_rate">25</property>
    <property name="meta.media.0.codec.pix_fmt">yuv420p</property>
    <property name="meta.media.0.codec.sample_aspect_ratio">1.06667</property>
    <property name="meta.media.0.codec.colorspace">601</property>
    <property name="meta.media.0.codec.color_trc">6</property>
    <property name="meta.media.0.codec.name">h264</property>
    <property name="meta.media.0.codec.long_name">H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10</property>
    <property name="meta.media.0.codec.bit_rate">0</property>
    <property name="meta.media.1.stream.type">audio</property>
    <property name="meta.media.1.codec.sample_fmt">fltp</property>
    <property name="meta.media.1.codec.sample_rate">48000</property>
    <property name="meta.media.1.codec.channels">2</property>
    <property name="meta.media.1.codec.name">aac</property>
    <property name="meta.media.1.codec.long_name">AAC (Advanced Audio Coding)</property>
    <property name="meta.media.1.codec.bit_rate">161625</property>
    <property name="meta.media.sample_aspect_num">16</property>
    <property name="meta.media.sample_aspect_den">15</property>
    <property name="aspect_ratio">1.06667</property>
    <property name="audio_index">1</property>
    <property name="video_index">0</property>
    <property name="mute_on_pause">1</property>
    <property name="mlt_service">avformat-novalidate</property>
  </producer>
  <playlist id="playlist0" autoclose="1">
    <entry producer="producer0" in="0" out="2147483646"/>
  </playlist>
  <tractor id="tractor0" title="127.0.0.1:1234?fifo_size=1000000&amp;overrun_nonfatal=1" global_feed="1" in="0" out="2160000">
    <track producer="playlist0"/>
  </tractor>
</mlt>

Thanks and Regards,

Jackson

[ddennedy]

There was a regression introduced on Oct. 30 for reading network streams. I noticed it recently in Shotcut as well, and I just fixed it in commit d25ba846d68f2d80a0d3d5a3bd48639784c8f48c.