search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.44k stars 130 forks source link

input.harbor TLS not exiting after first connection #4138

Open metaswirl opened 2 months ago

metaswirl commented 2 months ago

Description

I can connect to input.harbor via TLS, when I disconnect the source, nothing shows in the log, and when I try and re-connect this appears:

2024/09/20 18:15:49 [harbor:4] Harbor.Make(T).Mount_taken

I have tried various clients including liquidsoap output.icecast tls, mixxx and butt, all have the same issues.

Steps to reproduce

transport = http.transport.tls( certificate = "/opt/liquidsoap/cert/fullchain.pem", key = "/opt/liquidsoap/cert/privkey.pem" )

stream_live_tls = input.harbor("live-tls", port=9016, auth=auth, transport=transport, id="live-tls", icy=false, icy_metadata_charset="UTF-8", metadata_charset="UTF-8", replay_metadata=true, on_connect=live_connect_tls, on_disconnect=live_disconnect_tls, buffer=5.0, max=10.0)

Expected behavior

input.harbor TLS does not recognize when source has disconnected making it impossible to re-connect.

Liquidsoap version

Liquidsoap 2.2.5+dev
Copyright (c) 2003-2024 Savonet team
Liquidsoap is open-source software, released under GNU General Public License.
See <http://liquidsoap.info> for more information.

Liquidsoap build config

* Liquidsoap version  : 2.2.5+dev

 * Compilation options
   - Release build       : false
   - Git SHA             : (none)
   - OCaml version       : 5.2.0
   - OS type             : Unix
   - Libs versions       : angstrom=0.16.0 asn1-combinators=0.2.6 astring=0.8.5 base64=3.5.1 bigarray-compat=1.1.0 bigstringaf=0.10.0 bos=0.2.1 bytes=[distributed with OCaml 4.02 or above] ca-certs=v0.2.3 camlp-streams camomile.lib=2.0 cry=1.0.3 cstruct=6.2.0 ctypes=0.23.0 ctypes-foreign=0.23.0 ctypes.stubs=0.23.0 curl=0.9.2 domain-name=0.4.0 dtools=0.4.5 dune-build-info=3.16.0 dune-private-libs.dune-section=3.16.0 dune-site=3.16.0 dune-site.private=3.16.0 duppy=0.9.4 eqaf=0.9 eqaf.bigstring=0.9 eqaf.cstruct=0.9 ffmpeg-av=1.1.11 ffmpeg-avcodec=1.1.11 ffmpeg-avdevice=1.1.11 ffmpeg-avfilter=1.1.11 ffmpeg-avutil=1.1.11 ffmpeg-swresample=1.1.11 ffmpeg-swscale=1.1.11 fileutils=0.6.4 fmt=0.9.0 fpath=0.7.3 gen=1.1 gmap=0.3.0 hkdf=1.0.4 integers ipaddr=5.6.0 ladspa=0.2.2 liquidsoap-lang=2.2.5 liquidsoap-lang.console=2.2.5 liquidsoap_builtins liquidsoap_core liquidsoap_ffmpeg liquidsoap_ladspa liquidsoap_optionals liquidsoap_oss liquidsoap_runtime liquidsoap_srt liquidsoap_stereotool liquidsoap_taglib liquidsoap_tls logs=0.7.0 macaddr=5.6.0 magic-mime=1.3.1 menhirLib=20240715 metadata=0.3.0 mirage-crypto=0.11.3 mirage-crypto-ec=0.11.3 mirage-crypto-pk=0.11.3 mirage-crypto-rng=0.11.3 mirage-crypto-rng.unix=0.11.3 mm=0.8.5 mm.audio=0.8.5 mm.base=0.8.5 mm.image=0.8.5 mm.midi=0.8.5 mm.video=0.8.5 pbkdf pcre=7.5.0 posix-base=2.0.2 posix-socket=2.0.2 posix-socket.constants=2.0.2 posix-socket.stubs=2.0.2 posix-socket.types=2.0.2 ppx_string.runtime=v0.17.0 ptime=1.1.0 ptime.clock.os=1.1.0 re=1.12.0 rresult=0.7.0 sedlex=3.2 seq=[distributed with OCaml 4.07 or above] sexplib0=v0.17.0 srt=0.3.0 srt.constants=0.3.0 srt.stubs=0.3.0 srt.stubs.locked=0.3.0 srt.types=0.3.0 stdlib-shims=0.3.0 stereotool str=5.2.0 stringext=1.6.0 taglib=0.3.10 threads=5.2.0 tls=0.17.3 unix=5.2.0 uri=4.4.0 x509=0.16.5 zarith=1.14
   - architecture        : amd64
   - host                : x86_64-pc-linux-gnu
   - target              : x86_64-pc-linux-gnu
   - system              : linux
   - ocamlopt_cflags     : -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread
   - native_c_compiler   : gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread -D_FILE_OFFSET_BITS=64
   - native_c_libraries  : -lm -ldl -lpthread

 * Configured paths
   - mode              : default
   - standard library  : (set by dune-site)
   - scripted binaries : (set by dune-site)
   - rundir            : (set by dune-site)
   - logdir            : (set by dune-site)
   - camomile files    : (set by dune-site)

 * Supported input formats
   - MP3               : no (requires mad)
   - AAC               : no (requires faad)
   - Ffmpeg            : yes
   - Flac (native)     : no (requires flac)
   - Flac (ogg)        : no (requires ogg)
   - Opus              : no (requires opus)
   - Speex             : no (requires speex)
   - Theora            : no (requires theora)
   - Vorbis            : no (requires vorbis)

 * Supported output formats
   - FDK-AAC           : no (requires fdkaac)
   - Ffmpeg            : yes
   - MP3               : no (requires lame)
   - MP3 (fixed-point) : no (requires shine)
   - Flac (native)     : no (requires flac)
   - Flac (ogg)        : no (requires ogg)
   - Opus              : no (requires opus)
   - Speex             : no (requires speex)
   - Theora            : no (requires theora)
   - Vorbis            : no (requires vorbis)

 * Tags
   - Taglib (ID3 tags) : yes
   - Vorbis            : no (requires vorbis)

 * Input / output
   - ALSA              : no (requires alsa)
   - AO                : no (requires ao)
   - FFmpeg            : yes
   - GStreamer         : no (requires gstreamer)
   - JACK              : no (requires bjack)
   - OSS               : yes
   - Portaudio         : no (requires portaudio)
   - Pulseaudio        : no (requires pulseaudio)
   - SRT               : yes

 * Audio manipulation
   - FFmpeg            : yes
   - LADSPA            : yes
   - Lilv              : no (requires lilv)
   - Samplerate        : no (requires samplerate)
   - SoundTouch        : no (requires soundtouch)
   - StereoTool        : yes

 * Video manipulation
   - camlimages        : no (requires camlimages)
   - FFmpeg            : yes
   - frei0r            : no (requires frei0r)
   - ImageLib          : no (requires imagelib)
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * MIDI manipulation
   - DSSI              : no (requires dssi)

 * Visualization
   - GD                : no (requires gd)
   - Graphics          : no (requires graphics)
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * Additional libraries
   - FFmpeg filters    : yes
   - FFmpeg devices    : yes
   - inotify           : no (requires inotify)
   - irc               : no (requires irc-client-unix)
   - jemalloc          : no (requires jemalloc)
   - lastfm            : no (requires lastfm)
   - lo                : no (requires lo)
   - memtrace          : no (requires memtrace)
   - mem_usage         : no (requires mem_usage)
   - osc               : no (requires osc-unix)
   - ssl               : no (requires ssl)
   - tls               : yes
   - posix-time2       : no (requires posix)
   - windows service   : no (requires winsvc)
   - YAML support      : no (requires yaml)
   - XML playlists     : no (requires xmlplaylist)

 * Monitoring
   - Prometheus        : no (requires prometheus)

Installation method

From OPAM

Additional Info

No response

vitoyucepi commented 2 months ago

Hi @metaswirl,

I'm able to reproduce this problem. Also, I've found another problem when trying to connect to it.

So here's my reproduction script

  1. compose.yaml 
    services:
 ls22:
   image: savonet/liquidsoap:v2.2.5
   command: /tmp/test/main.liq
   volumes:
     - ./:/tmp/test/
   ports:
     - 8000:8000

  2. main.liq

    settings.log.level := 5

transport = http.transport.tls(
 certificate = "/tmp/test/cert.pem",
 key = "/tmp/test/key.pem",
)

live = input.harbor(
 "live",
 port=8000,
 buffer=3.,
 transport=transport,
)

output.dummy(live, fallible=true)
  3. openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -subj '/CN=localhost' -nodes;
chmod 0644 key.pem;
docker compose up;
  4. ffmpeg -re -f lavfi -i "sine=frequency=200:duration=2000" -b:a 320k -c:a mp3 -f mp3 -tls 1 icecast://source:hackme@localhost:8000/live

But that's not working properly, the stream doesn't even start. So I've configured reverse proxy and got the error.

Log ``` 2024/09/24 01:45:53 [input.harbor:5] Not ready: need more buffering (0/132300). 2024/09/24 01:46:13 [harbor:4] New client on port 8000: 172.18.0.1 2024/09/24 01:46:13 [harbor:4] Method: PUT, uri: /live, protocol: HTTP/1.1 2024/09/24 01:46:13 [harbor:4] Header: User-Agent, value: Lavf/61.1.100. 2024/09/24 01:46:13 [harbor:4] Header: Accept, value: */*. 2024/09/24 01:46:13 [harbor:4] Header: Expect, value: 100-continue. 2024/09/24 01:46:13 [harbor:4] Header: Connection, value: close. 2024/09/24 01:46:13 [harbor:4] Header: Host, value: 192.168.0.1:8080. 2024/09/24 01:46:13 [harbor:4] Header: Content-Type, value: audio/mpeg. 2024/09/24 01:46:13 [harbor:4] Header: Icy-MetaData, value: 1. 2024/09/24 01:46:13 [harbor:4] Header: Ice-Public, value: 0. 2024/09/24 01:46:13 [harbor:4] Client logged in. 2024/09/24 01:46:13 [harbor:4] PUT (source) request on /live. 2024/09/24 01:46:13 [harbor:4] Adding source on mountpoint "/live" with type "audio/mpeg". 2024/09/24 01:46:13 [harbor:5] Relaying HTTP/1.1. 2024/09/24 01:46:13 [decoder:4] Available decoders: 2024/09/24 01:46:13 [decoder:4] ffmpeg (priority: 10) 2024/09/24 01:46:13 [decoder:4] mad (priority: 1) 2024/09/24 01:46:13 [decoder:4] Selected decoder ffmpeg for mime-type audio/mpeg with expected content {audio=pcm(stereo)} 2024/09/24 01:46:13 [threads:4] Created thread "harbor source feeding" (2 total). 2024/09/24 01:46:13 [input.harbor:3] Decoding... 2024/09/24 01:46:14 [input.harbor:5] Not ready: need more buffering (57600/132300). ... 2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (123264/132300). 2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (125568/132300). 2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (127872/132300). 2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (130176/132300). 2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (131328/132300). 2024/09/24 01:46:15 [input.harbor:5] Got metadata at position 0: calling handlers... 2024/09/24 01:46:27 [input.harbor:4] End of track. 2024/09/24 01:46:27 [input.harbor:4] Buffer emptied, buffering needed. 2024/09/24 01:46:27 [input.harbor:5] Not ready: need more buffering (0/132300). 2024/09/24 01:46:27 [dummy:3] Source failed (no more tracks) stopping output... 2024/09/24 01:46:42 [harbor:4] New client on port 8000: 172.18.0.1 2024/09/24 01:46:42 [harbor:4] Method: PUT, uri: /live, protocol: HTTP/1.1 2024/09/24 01:46:42 [harbor:4] Header: User-Agent, value: Lavf/61.1.100. 2024/09/24 01:46:42 [harbor:4] Header: Accept, value: */*. 2024/09/24 01:46:42 [harbor:4] Header: Expect, value: 100-continue. 2024/09/24 01:46:42 [harbor:4] Header: Connection, value: close. 2024/09/24 01:46:42 [harbor:4] Header: Host, value: 192.168.0.1:8080. 2024/09/24 01:46:42 [harbor:4] Header: Content-Type, value: audio/mpeg. 2024/09/24 01:46:42 [harbor:4] Header: Icy-MetaData, value: 1. 2024/09/24 01:46:42 [harbor:4] Header: Ice-Public, value: 0. 2024/09/24 01:46:42 [harbor:4] Client logged in. 2024/09/24 01:46:42 [harbor:4] PUT (source) request on /live. 2024/09/24 01:46:42 [harbor:4] Adding source on mountpoint "/live" with type "audio/mpeg". 2024/09/24 01:46:42 [harbor:5] Relaying HTTP/1.1. 2024/09/24 01:46:42 [harbor:4] Harbor.Make(T).Mount_taken ```

If you need a quick fix, then set up a reverse proxy in front of liquidsoap and configure it to strip encryption in TCP mode. https://github.com/savonet/liquidsoap/discussions/3659#discussioncomment-8241421

toots commented 2 months ago

Thanks for the report, I will have a pass at it.

Is it also happening in 2.3.0-rc1 or main?

Also worth noting we do have SSL as well. A reverse proxy is never a bad idea if that suits your infrastructure.

metaswirl commented 2 months ago

Thank you Toots!

http.transport.ssl works without any issue! http.transport.tls is buggy.

vitoyucepi commented 2 months ago

Both http.transport.ssl and http.transport.tls have their own quirks. Ref: #3574