search

Polochon-street / blissify-rs

bliss bindings for MPD.
30 stars 6 forks source link

Trouble analyzing library #47

Open brawnswubb opened 10 months ago

brawnswubb commented 10 months ago

Hello,

Running blissify 0.3.8 on Arch Linux with a MPD library of over 300k songs

When I set log_level = "verbose" in mpd config I see that the output buffer is full when running blissify init 

Jan 07 23:08 : client: [4] opened from 127.0.0.1:43290
Jan 07 23:08 : client: [4] process command "list "file""
Jan 07 23:08 : client: error on client 4: Output buffer is full
Jan 07 23:08 : client: [4] command returned 0

In order for blissify to run successfully max_output_buffer_size has been set to a high value Prior to setting the arbitrarily high value, a Output buffer is full was observed at the 16384 and 32768 values. I set max_output_buffer_size to "323684" in order for it to run. Earlyoom kills programs once my current RAM is less than 1% available. Blissify is terminated after some time analyzing.

Is there a way to adjust what Blissify is requesting from the MPD server?

Polochon-street commented 10 months ago

Hi,

This should indeed not happen. I think part of it is due to the way rust-mpd handles the list / search for all your songs; I've opened a bug report, and most likely will have to change some things in blissify myself.

Thanks for reporting, I'm on it.

Polochon-street commented 5 months ago

Hi again! It seems that someone has the same problem as you, and I'm trying to address the root cause (which doesn't seem to be MPD / rust-MPD themselves). This branch https://github.com/Polochon-street/blissify-rs/pull/58 should fix the buffer size problem, but now I am trying to diagnose the OOM, which I can't seem to reproduce. Is your setup special in any way that could give us a hint on how to solve it?

Polochon-street commented 5 months ago

@brawnswubb now that I think about it - could you try to run RUST_LOG=debug blissify init --number-cores 1 path/to/library and tell me if it always fail on the same song? I'm thinking maybe a specific song is causing the problem :)

Polochon-street commented 5 months ago

Update, it seems like you can "fix" it by running the analyzer with a lower number of cores (see https://github.com/Polochon-street/blissify-rs/issues/56#issuecomment-2155276777). Maybe try with 2 or 4 cores, and see if it works? blissify init --number-cores 2 /path/to/library