search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.4k stars 130 forks source link

Replay gain is not ignored for `opus` by default. #2172

Closed vitoyucepi closed 2 years ago

vitoyucepi commented 2 years ago

Describe the bug If opus file contains replay gain metadata tag, then it will use it even if enable_replaygain_metadata() not used. I checked some other file formats and there's no such bug for them.

Logs ``` liquidsoap_1 | 2022/01/26 02:03:18 >>> LOG START liquidsoap_1 | 2022/01/26 02:03:16 [main:3] Liquidsoap 2.0.2 liquidsoap_1 | 2022/01/26 02:03:16 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.4.6 sedlex=2.3 menhirLib=20210419 curl=0.9.1 dtools=0.4.4 duppy=0.9.2 cry=0.6.5 mm=0.7.3 xmlplaylist=0.1.5 lastfm=0.3.3 ogg=0.7.0 ogg.decoder=0.7.0 vorbis=0.8.0 vorbis.decoder=0.8.0 opus=0.2.1 opus.decoder=0.2.1 speex=0.4.0 speex.decoder=0.4.0 mad=0.5.1 flac=0.3.0 flac.ogg=0.3.0 flac.decoder=0.3.0 dynlink=[distributed with Ocaml] lame=0.3.5 shine=0.2.2 frei0r=0.1.2 fdkaac=0.3.2 theora=0.4.0 theora.decoder=0.4.0 ffmpeg=1.1.1 bjack=0.1.6 alsa=0.3.0 ao=0.2.3 samplerate=0.1.6 taglib=0.3.7 ssl=0.5.9 magic=0.7.3 camomile=1.0.2 inotify=2.3 yojson=1.7.0 faad=0.5.0 soundtouch=0.1.9 portaudio=0.2.3 pulseaudio=0.1.4 ladspa=0.2.0 dssi=0.1.3 tsdl=v0.9.8 tsdl_ttf=0 tsdl-image=0.3.2 camlimages=4.2.6 cohttp-lwt-unix=2.5.5 prometheus-app=1.1 srt.constants=0.2.1 srt.types=0.2.1 srt.stubs=0.2.1 srt.stubs.locked=0.2.1 srt=0.2.1 lo=0.2.0 gd=1.0a5 liquidsoap_1 | 2022/01/26 02:03:16 [clock:3] Using builtin (low-precision) implementation for latency control liquidsoap_1 | 2022/01/26 02:03:17 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main. liquidsoap_1 | 2022/01/26 02:03:17 [frame:3] Video frame size set to: 1280x720 liquidsoap_1 | 2022/01/26 02:03:17 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples. liquidsoap_1 | 2022/01/26 02:03:17 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks. liquidsoap_1 | 2022/01/26 02:03:17 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks. liquidsoap_1 | 2022/01/26 02:03:18 [sandbox:3] Sandboxing disabled liquidsoap_1 | 2022/01/26 02:03:18 [video.converter:3] Using preferred video converter: ffmpeg. liquidsoap_1 | 2022/01/26 02:03:18 [audio.converter:3] Using samplerate converter: ffmpeg. liquidsoap_1 | 2022/01/26 02:03:18 [single_0:3] /var/liquidsoap/with_RG.opus is static, resolving once for all... liquidsoap_1 | 2022/01/26 02:03:18 [single_1:3] /var/liquidsoap/without_RG.opus is static, resolving once for all... liquidsoap_1 | 2022/01/26 02:03:18 [single_0:3] Prepared "/var/liquidsoap/with_RG.opus" (RID 0). liquidsoap_1 | 2022/01/26 02:03:18 [clock.main:3] Streaming loop starts in auto-sync mode liquidsoap_1 | 2022/01/26 02:03:18 [clock.main:3] Delegating synchronisation to CPU clock liquidsoap_1 | 2022/01/26 02:03:18 [switch_1:3] Switch to switch_0. liquidsoap_1 | 2022/01/26 02:03:18 [switch_0:3] Switch to single_0. liquidsoap_1 | [opus @ 0x7fa33000bb40] Could not update timestamps for skipped samples. liquidsoap_1 | [opus @ 0x7fa33000bb40] Could not update timestamps for discarded samples. liquidsoap_1 | 2022/01/26 02:03:28 [decoder:2] Decoding "/var/liquidsoap/with_RG.opus" ended: Ffmpeg_decoder.End_of_file. liquidsoap_1 | 2022/01/26 02:03:29 [single_0:3] Prepared "/var/liquidsoap/with_RG.opus" (RID 0). liquidsoap_1 | 2022/01/26 02:03:29 [single_1:3] Prepared "/var/liquidsoap/without_RG.opus" (RID 1). liquidsoap_1 | 2022/01/26 02:03:29 [switch_0:3] Switch to single_1 with forgetful transition. liquidsoap_1 | [opus @ 0x7fa330013880] Could not update timestamps for skipped samples. liquidsoap_1 | [opus @ 0x7fa330013880] Could not update timestamps for discarded samples. liquidsoap_1 | 2022/01/26 02:03:39 [decoder:2] Decoding "/var/liquidsoap/without_RG.opus" ended: Ffmpeg_decoder.End_of_file. liquidsoap_1 | 2022/01/26 02:03:40 [switch_0:3] Switch to single_0. liquidsoap_1 | [opus @ 0x7fa330054bc0] Could not update timestamps for skipped samples. ```

To Reproduce

  1. Get sample files

    • testfile_original.flac is the reference file
    • testfile_with_RG.opus file with replay gain metadata tag
      ffprobe testfile_with_RG.opus 
    ffprobe version n4.4.1 Copyright (c) 2007-2021 the FFmpeg developers
 built with gcc 11.1.0 (GCC)
 configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
 libavutil      56. 70.100 / 56. 70.100
 libavcodec     58.134.100 / 58.134.100
 libavformat    58. 76.100 / 58. 76.100
 libavdevice    58. 13.100 / 58. 13.100
 libavfilter     7.110.100 /  7.110.100
 libswscale      5.  9.100 /  5.  9.100
 libswresample   3.  9.100 /  3.  9.100
 libpostproc    55.  9.100 / 55.  9.100
[ogg @ 0x5637fc923c40] 1 bytes of comment header remain
Input #0, ogg, from 'testfile_with_RG.opus':
 Duration: 00:00:11.04, start: 0.000000, bitrate: 184 kb/s
 Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
   Metadata:
     R128_TRACK_GAIN : -5120

    • testfile_without_RG.opus file without tag
      ffprobe testfile_without_RG.opus 
    ffprobe version n4.4.1 Copyright (c) 2007-2021 the FFmpeg developers
 built with gcc 11.1.0 (GCC)
 configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
 libavutil      56. 70.100 / 56. 70.100
 libavcodec     58.134.100 / 58.134.100
 libavformat    58. 76.100 / 58. 76.100
 libavdevice    58. 13.100 / 58. 13.100
 libavfilter     7.110.100 /  7.110.100
 libswscale      5.  9.100 /  5.  9.100
 libswresample   3.  9.100 /  3.  9.100
 libpostproc    55.  9.100 / 55.  9.100
[ogg @ 0x559f3c9c5c40] 1 bytes of comment header remain
Input #0, ogg, from 'testfile_without_RG.opus':
 Duration: 00:00:11.04, start: 0.000000, bitrate: 184 kb/s
 Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp

  2. Check files with players
    • ffplay
    • vlc
    • others...

  3. Create config

    track_1 = single("/var/liquidsoap/testfile_with_RG.opus")
track_2 = single("/var/liquidsoap/testfile_without_RG.opus")
r = rotate([track_1, track_2])

output.file(%vorbis, "/tmp/test.ogg", r)
  4. Launch liquidsoap
  5. Wait for some time until both files are recorded to /tmp/test.ogg
  6. Listen /tmp/test.ogg
  7. Hear volume difference

Expected behavior Replay gain should be ignored by default in opus files.

Version details

Install method Deb package from liquidsoap ci artifacts at github

Common issues N/A

toots commented 2 years ago

Issue re-opened, see my rational here: https://github.com/savonet/liquidsoap/commit/acfaea458aa0f61ab4c67de72fbbd87349a5eb37#commitcomment-64698991

First, R128_TRACK_GAIN seems to relate to a different standard than replaygain: https://en.wikipedia.org/wiki/EBU_R_128, it's not sure if we should or should not support it as part of the replaygain stack.

Second, I haven't run the test yet but there really is nothing that we do to change the volume on these tracks. We ask the ffmpeg decoder to decode it and send it to the vorbis encoder as is. I'm pretty sure the R128_TRACK_GAIN does not even get passed to the encoder.

If I run those files through the replaygain peak detections, a different value is returned:

❯ ffmpeg -i /tmp/testfile_with_RG.opus -vn -filter replaygain -f null /dev/null
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_5 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[ogg @ 0x133808200] 1 bytes of comment header remain
Input #0, ogg, from '/tmp/testfile_with_RG.opus':
  Duration: 00:00:11.04, start: 0.000000, bitrate: 184 kb/s
  Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      R128_TRACK_GAIN : -5120
Stream mapping:
  Stream #0:0 -> #0:0 (opus (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to '/dev/null':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
    Metadata:
      R128_TRACK_GAIN : -5120
      encoder         : Lavc58.134.100 pcm_s16le
size=N/A time=00:00:11.00 bitrate=N/A speed= 242x
video:0kB audio:2069kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[Parsed_replaygain_0 @ 0x132e04dc0] track_gain = -3.81 dB
[Parsed_replaygain_0 @ 0x132e04dc0] track_peak = 0.619401
❯ ffmpeg -i /tmp/testfile_without_RG.opus -vn -filter replaygain -f null /dev/null
ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/4.4.1_5 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[ogg @ 0x142008200] 1 bytes of comment header remain
Input #0, ogg, from '/tmp/testfile_without_RG.opus':
  Duration: 00:00:11.04, start: 0.000000, bitrate: 184 kb/s
  Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
Stream mapping:
  Stream #0:0 -> #0:0 (opus (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to '/dev/null':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
    Metadata:
      encoder         : Lavc58.134.100 pcm_s16le
size=N/A time=00:00:11.00 bitrate=N/A speed= 260x
video:0kB audio:2069kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[Parsed_replaygain_0 @ 0x141105710] track_gain = -8.81 dB
[Parsed_replaygain_0 @ 0x141105710] track_peak = 1.101468

This is also confirmed by a visual inspection of the decoded wav files:

Screen Shot 2022-01-26 at 9 47 12 PM Screen Shot 2022-01-26 at 9 47 19 PM
smimram commented 2 years ago

Sorry about this, I have known for long that we did not properly apply replaygain for opus and read this bug report too quickly...

vitoyucepi commented 2 years ago

Update: After some testing with ffmpeg and other players.

For example if I set apply gain in playback settings, then player will change volume according to replay gain.

Without audio processing all music has equal volume. Anyway it's not clear if it's metadata related or not.

dFasdGii commented 2 years ago

@toots files in audacity are different because you need remove all gain info from opus file BEFORE encoding:

https://datatracker.ietf.org/doc/html/rfc7845 :

Players and media frameworks SHOULD apply it by default. If a player chooses to apply any volume adjustment or gain modification, such as the R128_TRACK_GAIN (see Section 5.2), the adjustment MUST be applied in addition to this output gain in order to achieve playback at the normalized volume.

A muxer SHOULD set this field to zero, and instead apply any gain prior to encoding, when this is possible and does not conflict with the user's wishes. A nonzero output gain indicates the gain was adjusted after encoding, or that a user wished to adjust the gain for playback while preserving the ability to recover the original signal amplitude.

https://github.com/Moonbase59/loudgain#how-i-handle-opus-opus-audio-files :

Encoders and muxes (should) set 'output_gain' to zero, so a non-zero 'output_gain' in the opus header is supposed to be a change after encoding/muxing.

Also, according to this info: https://wiki.xiph.org/OggOpus#Comment_Header when ReplayGain reading from file is disabled, music players must ignore both ReplayGain (R128_TRACK_GAIN) and Opus Header Gain. Don't know much about linux music players, but popular windows players like foobar2000 and AIMP works fine with this opus files, and if option "replaygain" is disabled, files sound the same, but if play files with RG through liquidsoap, they are quiet