Problem to stream videos to rtmp when using %video.copy and %audio.copy

[caf2022]

Describe the bug When I try to stream videos to rtmp server using the video and audio copy on encoder I got an error

To Reproduce

settings.log.file.path.set("/home/rtmp.log")
settings.init.allow_root.set(true)
settings.log.level.set(3)
settings.log.stdout.set(true)
settings.harbor.max_connections.set(1)
#settings.init.daemon.set(true)
settings.scheduler.fast_queues.set(0)
settings.harbor.bind_addrs.set(["0.0.0.0"])
settings.init.allow_root.set(true)
settings.video.converter.preferred.set('ffmpeg')
settings.request.metadata_decoders.set(["FFMPEG"])
settings.ffmpeg.log.level.set(1)
settings.ffmpeg.log.verbosity.set("error")

playlist = playlist(reload_mode="watch", loop=true, mode="normal", "/home/playlist.pls")

streaming = mksafe(playlist)

encoder = %ffmpeg(format="flv", %video.copy, %audio.copy)

output.url(fallible=true,url="rtmp://127.0.0.1:1935/test/test", encoder, streaming)

Error

At /home/autovj.liq, line 24, char 75-84:
output.url(fallible=true,url="rtmp://127.0.0.1:1935/test/test", encoder, streaming)

Error 5: this value has type
  source(audio=?internal(?A),...) (inferred at /root/.opam/4.14.0/lib/liquidsoap/share/liquidsoap/2.0.4/libs/switches.liq, line 16 char 2 - line 18 char 42)
but it should be a subtype of
  source(audio=ffmpeg.audio.copy,...)

At /home/autovj.liq, line 24, char 75-84:
output.url(fallible=true,url="rtmp://127.0.0.1:1935/test/test", encoder, streaming)

Error 5: this value has type
  source(, video=?internal(?A),...) (inferred at /root/.opam/4.14.0/lib/liquidsoap/share/liquidsoap/2.0.4/libs/switches.liq, line 16 char 2 - line 18 char 42)
but it should be a subtype of
  source(, video=ffmpeg.video.copy,...)

If I use the encoder options works fine like: encoder = %ffmpeg(format="flv", %video(codec="libx264", b="3500k", preset="ultrafast"), %audio(codec="aac", samplerate=44100, b="96k"))

[vitoyucepi]

Hi @caf2022, mksafe() produces frames in internal format, they could not be copied with the copy codec. You should use ffmpeg.encode.audio_video to encode blank().

e1 = %ffmpeg(
  %video(codec="libx264", b="3500k", preset="ultrafast"),
  %audio(codec="aac", samplerate=44100, b="96k")
)
b = ffmpeg.encode.audio_video(e1, blank())
streaming = fallback([playlist, b])

[caf2022]

Hi @vitoyucepi first of all, thanks for the help, I make the changes but the problem continue:

At /home/autovj.liq, line 23, char 35-42:
b= ffmpeg.encode.audio_video(e1, blank())

Error 5: this value has type
  format(audio=ffmpeg.audio.copy(_),...) (inferred at /home/autovj.liq, line 21, char 10-57)
but it should be a subtype of
  format(audio=pcm(_),...)

The problem is only if I use the video and audio copy:

e1 = %ffmpeg(format="flv", %video.copy, %audio.copy)

[vitoyucepi]

Maybe I'm missing something, but liquidsoap starts with this script

playlist = playlist(reload_mode="watch", loop=true, mode="normal", "/home/playlist.pls")

e1 = %ffmpeg(
  %video(codec="libx264", b="3500k", preset="ultrafast"),
  %audio(codec="aac", samplerate=44100, b="96k")
)
b = ffmpeg.encode.audio_video(e1, blank())
streaming = fallback([playlist, b])

encoder = %ffmpeg(format="flv", %video.copy, %audio.copy)

output.url(fallible=true,url="rtmp://127.0.0.1:1935/test/test", encoder, streaming)

[caf2022]

@vitoyucepi Sorry, it was my mistake, I didn't understand that e1 was to create the blank video and I was using it as the final encoder, now is working fine.

But I have a last question if you can help a little more, the liquidsoap starts and run fine but, if I set to run with daemon mode, it starts but not work normal, nothing happen, the process still running but not broadcast to the rtmp server and the logs not show any error, this only occours if I set to run with daemon mode. The actual code is:

settings.log.file.path.set("/home/autovj.log")
settings.init.daemon.pidfile.path.set("/home/autovj.pid")
settings.init.allow_root.set(true)
settings.log.level.set(3)
settings.log.stdout.set(true)
settings.harbor.max_connections.set(1)
settings.init.daemon.set(true)
settings.scheduler.fast_queues.set(0)
settings.harbor.bind_addrs.set(["0.0.0.0"])
setenv("TZ", "America/Sao_Paulo")
settings.video.converter.preferred.set('ffmpeg')
settings.request.metadata_decoders.set(["FFMPEG"])
settings.ffmpeg.log.level.set(1)
settings.ffmpeg.log.verbosity.set("error")

playlist = playlist(reload_mode="watch", loop=true, mode="normal", "/home/playlist.pls")

default_encoder = %ffmpeg(%video(codec="libx264", b="512k", preset="ultrafast"),%audio(codec="aac", samplerate=44100, b="32k"))

default = ffmpeg.encode.audio_video(default_encoder, blank())

streaming = fallback([playlist, default])

encoder = %ffmpeg(format="flv", %video.copy, %audio.copy)

output.url(fallible=true,url="rtmp://127.0.0.1:1935/test/test", encoder, streaming)

The file log show this:

2024/07/06 18:35:01 >>> LOG START
2024/07/06 18:35:00 [main:3] Liquidsoap 2.0.4
2024/07/06 18:35:00 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.5.0 sedlex=3.2 menhirLib=20211128 curl=0.9.2 dtools=0.4.4 duppy=0.9.2 cry=0.6.7 mm=0.8.5 xmlplaylist=0.1.5 ogg=0.7.4 ogg.decoder=0.7.4 vorbis=0.8.0 vorbis.decoder=0.8.0 opus=0.2.2 opus.decoder=0.2.2 speex=0.4.1 speex.decoder=0.4.1 mad=0.5.2 flac=0.3.0 flac.ogg=0.3.0 flac.decoder=0.3.0 dynlink=[distributed with Ocaml] lame=0.3.6 frei0r=0.1.2 fdkaac=0.3.2 theora=0.4.0 theora.decoder=0.4.0 ffmpeg=1.1.4 samplerate=0.1.6 taglib=0.3.9 ssl=0.5.9 camomile=1.0.2 faad=0.5.0 ladspa=0.2.2
2024/07/06 18:35:00 [clock:3] Using builtin (low-precision) implementation for latency control
2024/07/06 18:35:01 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2024/07/06 18:35:01 [frame:3] Video frame size set to: 1280x720
2024/07/06 18:35:01 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
2024/07/06 18:35:01 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2024/07/06 18:35:01 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
2024/07/06 18:35:01 [sandbox:3] Sandboxing disabled
2024/07/06 18:35:01 [video.converter:3] Using preferred video converter: ffmpeg.
2024/07/06 18:35:01 [audio.converter:3] Using samplerate converter: ffmpeg.
2024/07/06 18:35:01 [video.add_text:3] Using native implementation
2024/07/06 18:35:01 [clock.main:3] Streaming loop starts in auto-sync mode
2024/07/06 18:35:01 [clock.main:3] Delegating synchronisation to CPU clock
2024/07/06 18:35:02 [playlist.pls:3] Prepared "/home/videos/demo1.mp4" (RID 1).
2024/07/06 18:35:02 [switch_0:3] Switch to playlist.pls.

[toots]

@caf2022 that fallback might be track sensitive but your blank has no tracks.

Besides, using a blank encoder defeats the purpose of the copy since you're still using your CPU to encode blank (which can actually be expensive).

I'd recommend:

1. First try with fallible=true and remove mksafe all together.
2. If needed, consider using a pre-encoded blank file as a fallsafe single.