search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.4k stars 129 forks source link

Liquidsoap 1.3.4 win64 can't read non-english files and metadata #627

Closed dFasdGii closed 3 years ago

dFasdGii commented 6 years ago

Liquidsoap build v 1.3.4 (win64) can't read folders, filenames, and metadata with non-english characters, e.g. Russian (Cyrillic), characters from swedish and german language ö å Ä. Even if file names in english, metadata also broken. Before I use 1.0.1 in that build filepath's and metadata work correctly. log: 2018/09/16 21:58:00 [decoder:3] Unable to decode "E:/music/metal/\208\145\208\190\209\143\208\189\208\190\208\178 \208\147\208\184\208\188\208\189/2009 - \208\173\209\133\208\190 \208\180\209\128\208\181\208\178\208\189\208\190\209\129\209\130\208\184/07 \208\159\209\128\208\190\209\130\208\184\208\178\208\190\209\129\209\130\208\190\209\143\208\189\208\184\208\181.mp3" as {audio=2;video=0;midi=0}!

My config.liq is simple:

set("log.file",true)
set("log.stdout",true)
set("log.level",3)
set("server.telnet",false)
set("server.telnet.bind_addr","127.0.0.1")
set("server.telnet.port",1234)

nonstop = playlist(reload=36000, "E:/music/metal")
radio = nonstop

output.icecast(%vorbis(samplerate=44100, channels=2, quality=0.6),
id="myid",
host="localhost",
port=80,
name="Stream Name",
description="Description",
genre="music",
url="http://www.liquidsoap.info",
public=false,
password="hackme",
fallible=true,
mount="mount.ogg",
encoding="UTF-8",
radio)
toots commented 6 years ago

Thanks for reporting this, I'll try to look at it asap.

toots commented 6 years ago

Ok, thank you for reporting that. These were issues with recent changes in the OCaml compiler when dealing with unicode filenames on windows. I've uploaded two fixed builds for win64 and win32. Gonna close now but feel free to reopen if it happens again, which I believe may happen..

MadLostSoul commented 3 years ago

I have the problem described in v1.4.4. I wanted to update from v1.1.1. My configuration:

set("log.file",true)
set("log.file.path", "ls2_service.log")
set("log.level",666)

# Safe Playlist
safe = mksafe(playlist("D:\\liquidsoap\\playlists\\failsafe.m3u", mime_type = "application/x-mpegURL", mode = "normal"))
safe = amplify(2.,override="replaygain_track_gain",safe)

# An output to the local soundcard
output.ao(fallible=true,safe)

Content of the (shortened) log file: Please have a look at the file names in the decoder section.

2021/02/28 01:51:59 [failsafe(dot)m3u:3] Loading playlist...
2021/02/28 01:51:59 [failsafe(dot)m3u:3] Playlist treated as format application/x-mpegURL
2021/02/28 01:51:59 [failsafe(dot)m3u:3] Successfully loaded a playlist of 24 tracks.
2021/02/28 01:51:59 [failsafe(dot)m3u:4] Content kind is {audio=2;video=0;midi=0}.
2021/02/28 01:51:59 [failsafe(dot)m3u:4] Activations changed: static=[], dynamic=[mksafe:amplify_8501:ao:ao].
2021/02/28 01:51:59 [failsafe(dot)m3u:5] Queue is empty!
2021/02/28 01:51:59 [failsafe(dot)m3u:5] Failed to prepare track: no file.
2021/02/28 01:51:59 [decoder:4] Trying method "MAD" for "D:\\liquidsoap\\playlists\\failsafe\\(Die \196rzte)-M\228nner haben kein Gehirn (Vorlesungen zum Buch)\\01-(Die \196rzte)-Nackig vorm Fahrstuhl.mp3"...
2021/02/28 01:51:59 [decoder:4] Decoder "MAD" failed on "D:\\liquidsoap\\playlists\\failsafe\\(Die \196rzte)-M\228nner haben kein Gehirn (Vorlesungen zum Buch)\\01-(Die \196rzte)-Nackig vorm Fahrstuhl.mp3": Mad.Openfile_error("No such file or directory")!
2021/02/28 01:52:02 [failsafe(dot)m3u:5] Queue is empty!
2021/02/28 01:52:02 [failsafe(dot)m3u:5] Failed to prepare track: no file.
2021/02/28 01:52:03 [failsafe(dot)m3u:4] Activations changed: static=[], dynamic=[].
2021/02/28 01:52:03 [source:4] Source failsafe(dot)m3u gets down.
2021/02/28 01:52:03 [failsafe(dot)m3u:4] Waiting for feeding task to stop...
2021/02/28 01:52:03 [failsafe(dot)m3u:4] Cleaning up request queue...

The m3u files are created with MP3Tag in ANSI format (Windows CR LF) corresponding to Windows 1252.

#EXTM3U
#EXTINF:240,Die Ärzte - Nackig vorm Fahrstuhl
D:\liquidsoap\playlists\failsafe\(Die Ärzte)-Männer haben kein Gehirn (Vorlesungen zum Buch)\01-(Die Ärzte)-Nackig vorm Fahrstuhl.mp3
toots commented 3 years ago

Hi @MadLostSoul. What version of ocaml-mad are you using?

MadLostSoul commented 3 years ago

Hi @toots,

I use Liquidsoap in the Win-x64 version.

[main:3] Liquidsoap 1.4.4
[main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.4.6 sedlex=2.2 menhirLib=20181113 dtools=0.4.2 duppy=0.9.0 cry=0.6.5 mm=0.5.0 xmlplaylist=0.1.5 lastfm=0.3.2 ogg=0.5.2 vorbis=0.7.1 opus=0.1.3 speex=0.2.1 mad=0.4.6 flac=0.1.7 flac.ogg=0.1.7 dynlink=[distributed with Ocaml] lame=0.3.4 gstreamer=0.3.0 fdkaac=0.3.1 theora=0.3.1 ao=0.2.1 samplerate=0.1.5 taglib=0.3.5 ssl=0.5.9 camomile=1.0.2 yojson=1.7.0 faad=0.4.0 portaudio=0.2.1 srt.types=0.1.1 srt.stubs=0.1.1 srt.stubs=0.1.1 srt=0.1.1 winsvc=1.0.0
[gstreamer.loader:3] Loaded GStreamer 1.16.2 0
[frame:3] Using 44100Hz audio, 25Hz video, 44100Hz master.

It works with m3u8 files but my php code expects ANSI or Win1252 playlists ... I hope you can help me.

toots commented 3 years ago

I'll have a look as soon as time allows, thanks for reporting.

toots commented 3 years ago

What encoding are you using for the filenames? The ocaml-mad code expects UTF8 for filenames: https://github.com/savonet/ocaml-mad/commit/6a995391002fa87afbb2bd62ceba0303ff78835b#diff-32adc6704d55f1d27695b0f2ff5c0fc07eab5ed4e3d60866ed9ec85ca8eac888R167

MadLostSoul commented 3 years ago

As already mentioned, I use ANSI or Win1252 for the corresponding filelis, which has always worked extremely well so far. Since which version does the MAD decoder need UTF8 filelists? Do I understand correctly that I have to create all filelists in the UTF8 form type because LquidSoap no longer decodes ANSI filelists correctly?

toots commented 3 years ago

Internally, liquidsoap uses UTF8 so it makes sense that ocaml-mad would also expect UTF8 character for its file names. This started with version 0.4.6, about 3 years ago.

We have to pick an encoding and stick with it. At the moment, UTF8 appears to be the most common so that should be the one that will be convenient for most users.

I would recommend to encode your playlists in UTF8 as well. If that is not possible, then we provide the function string.recode that you might be able to use inside your scripts to re-encode your filenames in UTF8:

string.recode Convert a string. Effective only if Camomile is enabled.

Type: (?in_enc : string, ?out_enc : string, string) -> string

Arguments: in_enc (of type string, which defaults to ""): Input encoding. Autodetected if empty. out_enc (of type string, which defaults to "UTF-8"): Output encoding. (unlabeled) (of type string)

MadLostSoul commented 3 years ago

I would recommend to encode your playlists in UTF8 as well. If that is not possible, then we provide the function string.recode that you might be able to use inside your scripts to re-encode your filenames in UTF8:

It is actually better to change my playlists. Thanks for the hints, I looked at them all and therefore my decision to encode the playlists in UTF8. Thank you for your commitment and support! --- Sorry for my "GoogleTranslateEnglish", I hope I expressed myself correctly.