After some time running the `reload_mode="watch"` stop working

[caf2022]

Hi, after some tiem running the playlist reload_mode="watch" stop working and if the playlsit file is changed, the liquidsoap won't detect, but if I stop and start the liquidsoap, back to work.

I'm using a playlist schedule, that copy the new playlist into a current playlist and need that the liquidsoap detect the file change and do the reload but only work.

[caf2022]

Is anyone going through this too? It is happening in several instances, now it happened in one instance, I turned off and on the liquidsoap and it returned to recognize changes in the playlist file, but after a while it stops again.

The logs must show this, but when stop watching not show and the playlist changes wont work:

2022/06/18 10:36:04 [playlist.pls:3] Reloading playlist with URI /home/playlist.pls.

[toots]

Hi,

What method are you using to track changes? inotify or the native one?

[caf2022]

I don't know if I'm doing it the right way, what I do is the following, my schedules execute a command that copies the new playlist to the file I used initially when turning on liquidsoap with simples "cp" linux command:

cp new.pls default.pls

playlist = playlist(reload_mode="watch", loop=true, mode=playlist_modo, "/home/default.pls")

So I expected that with that liquidsoap would reload the new content.

[toots]

Could you share the first lines of your log? It should tell us if you're using inotify or the native mechanism.

[caf2022]

2022/07/13 12:31:12 >>> LOG START
2022/07/13 12:31:00 [main:3] Liquidsoap 2.0.4
2022/07/13 12:31:00 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.5.0 sedlex=2.6 menhirLib=20211128 curl=0.9.2 dtools=0.4.4 duppy=0.9.2 cry=0.6.7 mm=0.8.0 xmlplaylist=0.1.5 ogg=0.7.1 ogg.decoder=0.7.1 vorbis=0.8.0 vorbis.decoder=0.8.0 opus=0.2.1 opus.decoder=0.2.1 speex=0.4.1 speex.decoder=0.4.1 mad=0.5.2 flac=0.3.0 flac.ogg=0.3.0 flac.decoder=0.3.0 dynlink=[distributed with Ocaml] lame=0.3.6 frei0r=0.1.2 fdkaac=0.3.2 theora=0.4.0 theora.decoder=0.4.0 ffmpeg=1.1.3 samplerate=0.1.6 taglib=0.3.9 camomile=1.0.2 faad=0.5.0 ladspa=0.2.2
2022/07/13 12:31:01 [clock:3] Using builtin (low-precision) implementation for latency control
2022/07/13 12:31:11 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
2022/07/13 12:31:11 [frame:3] Video frame size set to: 1280x720
2022/07/13 12:31:11 [frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
2022/07/13 12:31:11 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2022/07/13 12:31:11 [frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
2022/07/13 12:31:12 [sandbox:3] Sandboxing disabled
2022/07/13 12:31:12 [video.converter:3] Using preferred video converter: ffmpeg.
2022/07/13 12:31:12 [audio.converter:3] Using samplerate converter: ffmpeg.
2022/07/13 12:31:12 [harbor:3] Adding mountpoint '/' on port 38116
2022/07/13 12:31:12 [clock.main:3] Streaming loop starts in auto-sync mode
2022/07/13 12:31:12 [clock.main:3] Delegating synchronisation to CPU clock
2022/07/13 12:31:12 [video.add_text:3] Using native implementation
2022/07/13 12:31:12 [mksafe:3] Switch to safe_blank.
2022/07/13 12:31:12 [switch_4:3] Switch to blank_2.
2022/07/13 12:31:12 [switch_5:3] Switch to blank_3.
2022/07/13 12:31:12 [metadata.map_6:3] Inserting missing metadata.
2022/07/13 12:31:12 [Teste:3] Connecting mount sid#1 for @127.0.0.1...
2022/07/13 12:31:12 [Teste:3] Connection setup was successful.
2022/07/13 12:31:12 [default.pls:3] Prepared "/home/teste.mp3" (RID 1).
2022/07/13 12:31:12 [mksafe:3] Switch to gerenciador_sources with transition.
2022/07/13 12:31:12 [gerenciador_sources:3] Switch to dj_playlist_fallback.
2022/07/13 12:31:12 [dj_playlist_fallback:3] Switch to crossfade_0.
2022/07/13 12:31:12 [gerenciador_sources_pedidos:3] Switch to default.pls.
2022/07/13 12:31:12 [lang:3] Current Song(manual): Teste

[toots]

Thanks for this. I looked at the code and I'm not sure yet what could make this happen. One thing that could help is any kind of log (with log.level.set(4)) around the time the file watcher stopped working. We're looking at perhaps a error being raised.

Also, some more details on your script would help.

[caf2022]

Using the inotify is better then the native method?

[toots]

It's a different implementation so it's important to know which one you're using before investigating. The inotify library gives us callback for when the file is changed while the mtime (native) implementation is checking every second if the last modified time on the file has been changed.

[caf2022]

I don't have the opam-inotify installed then I believe that is using the native implementation. I will try to get more details with log.level.set(4)

[brenc]

We're seeing this too with LS 2.1.0 Docker. In fact I remember having this issue with 1.x so I just switched to reloading the playlists once per day.

We have a process that regenerates the playlists once a day. It writes the file to a temp directory first then copies over the old playlist file. LS will reload this file exactly one time and never again after that, no matter how many times the playlist gets updated.

[main:3] Liquidsoap 2.1.0
[main:3] Using: bytes=[distributed with OCaml 4.02 or above] posix-time2=2.0.0 pcre=7.5.0 sedlex=3.0 menhirLib=20220210 curl=0.9.2 memtrace=v0.2.2 mem_usage=0.0.4 dtools=0.4.4 duppy=0.9.2 cry=0.6.8 mm=0.8.1 xmlplaylist=0.1.5 lastfm=0.3.3 ogg=0.7.2 ogg.decoder=0.7.2 vorbis=0.8.1 vorbis.decoder=0.8.1 opus=0.2.2 opus.decoder=0.2.2 speex=0.4.1 speex.decoder=0.4.1 mad=0.5.2 flac=0.3.0 flac.ogg=0.3.0 flac.decoder=0.3.0 dynlink=[distributed with Ocaml] lame=0.3.6 shine=0.2.3 frei0r=0.1.2 fdkaac=0.3.2 theora=0.4.0 theora.decoder=0.4.0 ffmpeg=1.1.4 bjack=0.1.6 alsa=0.3.0 ao=0.2.4 samplerate=0.1.6 taglib=0.3.9 ssl=0.5.9 magic=0.7.3 camomile=1.0.2 inotify=2.3 faad=0.5.2 soundtouch=0.1.9 portaudio=0.2.3 pulseaudio=0.1.5 ladspa=0.2.2 dssi=0.1.5 tsdl=v0.9.9 tsdl-ttf=0.3.2 tsdl-image=0.3.2 camlimages=4.2.6 camlimages.freetype=5.0.4 cohttp-lwt-unix=5.0.0 prometheus-app=1.2 srt.constants=0.2.2 srt.types=0.2.2 srt.stubs=0.2.2 srt.stubs.locked=0.2.2 srt=0.2.2 lo=0.2.0 gd=1.0a5
[clock:3] Using native (high-precision) implementation for latency control
[frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
[frame:3] Video frame size set to: 1280x720
[frame:3] Frame size must be a multiple of 1764 ticks = 1764 audio samples = 1 video samples.
[frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
[frame:3] Frames last 0.04s = 1764 audio samples = 1 video samples = 1764 ticks.
[sandbox:3] Sandboxing disabled
[video.converter:3] Using preferred video converter: ffmpeg.
[audio.converter:3] Using samplerate converter: libsamplerate.
[clock.main:3] Streaming loop starts in auto-sync mode
[clock.main:3] Delegating synchronisation to CPU clock
[video.text:3] Using camlimages implementation

Nothing stands out to me with log.level.set(4). I see the playlists get reloaded exactly once on the first update, but then the logs are completely silent after that.

If I manually edit the playlist file, LS will reload it on every save. If I generate a new playlist and copy over the old one, LS will reload it once. If I then edit the updated playlist file and save it, LS does not reload it again.

It appears that copying over the old playlist file breaks watch mode.

[smimram]

Thanks for the further analysis. From the log you are using inotify, which watches for a file, even if it changes its name. This means that if you replace it by another file, it is not the same file anymore and thus is should not be watched anymore... I think the easy solution there is to do something like

cat new_playlist > playlist

instead of

mv new_playlist playlist

unless this is really a problem for you?

[brenc]

That does appear to work! I'm using Node.js 16 so I changed:

fs.rename(playlistTempPath, playlistFilePath, function (err) {

To:

fs.copyFile(playlistTempPath, playlistFilePath, function (err) {

[brenc]

By the way, this only appears to work on a local filesystem - not when the playlists are on NFS. I'll have to switch back to timed reload, which is fine by me. For local playlist files copying over the old file consistently triggers a reload though.