search

savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.39k stars 126 forks source link

long time for start LiquidSoap - 20 second #2906

Open HDSVR opened 1 year ago

HDSVR commented 1 year ago

Hello im install LiquidSoap 2.1.3 from OCAML, hardware nano pi neo 512 (Armbian arm, allwinner h3 cortex a7) Create simple script play one folder to Alsa sound and start script . long time for start playing 2023/02/17 09:37:01 [clock:3] Using builtin (low-precision) implementation for late ncy control 21 second load this module ..... it/s very loong (( How fix this time

LiquidSoap 1.4.3 load my simple script playing 2 second

LOGG

2023/02/17 09:37:01 [main:3] Liquidsoap 2.1.3
init(0):init-start                         @start-triggers
init(0):init-start                         @stop-triggers
2023/02/17 09:37:01 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pc                                                                                                                                                          re=7.5.0 sedlex=3.0 menhirLib=20220210 curl=0.9.2 uri=4.2.0 dtools=0.4.4 duppy=0.9.                                                                                                                                                          2 cry=0.6.7 mm=0.8.1 mad=0.5.2 dynlink=[distributed with Ocaml] lame=0.3.6 fdkaac=0                                                                                                                                                          .3.3 ffmpeg=1.1.7 alsa=0.3.0 taglib=0.3.10 camomile=1.0.2
init(0):init-start                         @stop
2023/02/17 09:37:01 [clock:3] Using builtin (low-precision) implementation for late                                                                                                                                                          ncy control
init(0):init-start                         @return
2023/02/17 09:37:22 [frame:3] Using 44100Hz audio, 25Hz video, 44100Hz main.
init(0):init-start                         @stop-depends
2023/02/17 09:37:22 [frame:3] Video frame size set to: 1280x720
init(0):init-start                         @start-atom

Many Sanks

toots commented 1 year ago

Hi @HDSVR and thanks for reporting. We are aware of the issue. We have been shifting a lot of our internal operators to the standard script library and also expanded it a lot. This is giving the software a lot of flexibility but also means that the runtime cost of typechecking all this script code has increased.

We are planning to tackle this with first a cached of the resolved typechecks and then as time allows an optimization of the abstract syntactic tree so that only the code you effectively use in your script is loaded and executed.

We will keep you posted of our progress and close this issue when the launch time has been reduced back to an acceptable one.

Meanwhile, while this does not seem to be your case here, I would like to also mention the fact a custom liquidsoap build can help, especially if loaded with only the features you need. For instance, LADSPA plugins can take a lot of time to load at startup and etc.

gilou commented 7 months ago

Hi, Happy New Year fellow Liq users and devs ;)

I'm finally migrating my last instances to liq 2.2.3, and for my last 1.4 setup, if it doesn't get better, I'll either have to try to build a specific liquidsoap (maybe following #3609 not really funny to me, and pretty much a loss in flexibility…) or … basically set all the monitoring to actually wait 10s before starting actually reporting anything, which… feels lame. Also, I will eventually need LADSPA on that platform, but I still can trim things down, but it's sad.

Running Liq 2.2.3 from github actions, on Debian 12, I get:

2024/01/05 04:23:35 >>> LOG START                                       
2024/01/05 04:23:25 [ffmpeg.filter.bitstream:3] No valid mode found for filter pgs_frame_merge!
2024/01/05 04:23:26 [main:3] Liquidsoap 2.2.3                   
[…]
2024/01/05 04:23:26 [clock:3] Using native (high-precision) implementation for latency control
2024/01/05 04:23:35 [main:3] Standard library loaded in 8.75 seconds.
[…]
2024/01/05 04:23:35 [startup:3] DSSI plugins registration: 0.00s
2024/01/05 04:23:35 [startup:3] FFmpeg filters registration: 0.04s
2024/01/05 04:23:35 [startup:3] FFmpeg bitstream filters registration: 0.00s
2024/01/05 04:23:35 [startup:3] Lilv plugins registration: 0.00s
2024/01/05 04:23:35 [startup:3] Frei0r plugin registration: 0.00s
2024/01/05 04:23:35 [startup:3] LADSPA plugins registration: 0.06s
2024/01/05 04:23:35 [startup:3] Typechecking: 8.27s
2024/01/05 04:23:35 [startup:3] Evaluation: 0.02s
2024/01/05 04:23:35 [startup:3] Typechecking: 0.05s
2024/01/05 04:23:35 [startup:3] Evaluation: 0.00s
2024/01/05 04:23:35 [startup:3] Typechecking: 0.05s
2024/01/05 04:23:35 [startup:3] Evaluation: 0.01s
toots commented 7 months ago

Hi @gilou. We're aware of the issue and are planning to work on it.

Meanwhile, have you tried using the --unsafe CLI option? It disables some type checks that should be safe to ignore if you've tested your script before and results here in a 73% reduction of the load time, from 2.60s down to 0.47s..

gilou commented 7 months ago

Oh yeah. That's a good workaround, as I can typecheck on the CI, and won't be too much impacted if there's an actual issue… That's a lot better, though it sounds scary:

2024/01/05 17:35:49 >>> LOG START                                       
2024/01/05 17:35:48 [main:3] Standard library loaded in 1.75 seconds.
2024/01/05 17:35:49 [startup:3] DSSI plugins registration: 0.00s
2024/01/05 17:35:49 [startup:3] FFmpeg filters registration: 0.04s
2024/01/05 17:35:49 [startup:3] FFmpeg bitstream filters registration: 0.00s
2024/01/05 17:35:49 [startup:3] Lilv plugins registration: 0.00s
2024/01/05 17:35:49 [startup:3] Frei0r plugin registration: 0.00s
2024/01/05 17:35:49 [startup:3] LADSPA plugins registration: 0.00s
2024/01/05 17:35:49 [startup:3] Typechecking: 1.22s
2024/01/05 17:35:49 [startup:3] Evaluation: 0.02s
2024/01/05 17:35:49 [startup:3] Typechecking: 0.08s
2024/01/05 17:35:49 [startup:3] Evaluation: 0.01s
2024/01/05 17:35:49 [startup:3] Typechecking: 0.01s
2024/01/05 17:35:49 [startup:3] Evaluation: 0.01s

Also I noticed that it helps for -h (which may be automatically "unsafe" to gain speed, online help is really helpful to me…):

streams@stream:~/liq$ time liquidsoap -h output >/dev/null

real    0m11.852s
user    0m9.380s
sys     0m0.208s
streams@stream:~/liq$ time liquidsoap --unsafe -h output >/dev/null

real    0m3.233s
user    0m2.250s
sys     0m0.326s
streams@stream:~/liq$
toots commented 7 months ago

Excellent suggestion. You get your personal mention in the next changelog: https://github.com/savonet/liquidsoap/commit/295a01ecca7e70cafca2dd6a509c289ad7e0b411 😄

smimram commented 7 months ago

@toots are you sure that the change is innocuous? An easy way to make sure of this is to generate the whole markdown doc with and without your patch and make a diff. I fear that some types will be less precise in unsafe mode...

toots commented 7 months ago

You're right @smimram I should have tested it. Looks like the file is the same!

smimram commented 7 months ago

Great thanks! Last picky comment: could we automate this check to ensure that it remains this way? It's important that we get the right documentation and it would serve as a good test for unsafe...

toots commented 7 months ago

Sadly I have had to revert the optimization. Turned out that the returned types were indeed incomplete when using --unsafe to generate API doc..

smimram commented 7 months ago

Too bad but this is also what I remembered from my last try to do this...