Bulk Search - Taking a Long Time to Process, 35x longer than LSP 3 V0.6

[ngs428]

Foobar 2.1.5 c64, OpenLyrics 1.11

I have been testing the bulk search feature a bit and the results returned are accurate, it takes a while (35x longer) to search for the lyrics in bulk in OpenLyrics vs Lyrics Show Panel.

I only have 4 active sources as shown below, but a search on 11 songs from a popular album takes 1 min 51 seconds. I tried just having Musixmatch as the only source, still takes that long. Using the bulk feature in Lyrics Show Panel 3 V0.6, it takes 3.24 seconds to search the same 11 files and return the same results, all synced files from Musixmatch.

OpenLyrics Sources:

OpenLyrics Search results in 1 min 51 seconds:

LSP Sources:

LSP Search results in 3.24 seconds

[jacquesh]

This is expected behaviour. To avoid spamming sources too hard (which bulk search would otherwise make very easy), when doing a bulk search we insert an artificial delay between tracks if an internet source actually got queried (so not if you already had lyrics saved locally for that track).

The idea is that bulk search is both rare and not urgent: on occasion you do a handful of bulk searches for all the music you don't already have lyrics for, which can happily continue in the background while you do other things. If you have an urgent need for a particular track's lyrics then bulk search is simply the wrong tool to be using.

[ngs428]

I only keep LSP installed for its fast bulk search. I haven’t had any issues in 10+ years of use of spamming sources too much. That includes processing around 20,000+ tracks all at one time, took hours straight if I remember right.

In my use case, bulk search would be the main method (if it were faster). Why would a user not do a bulk search for anything more than say 5 tracks. If the program will do everything on its own then that is the way to go.

I have not seen issues with spamming and use it as my main search method (using LSP). I’d love to dump that one to the side, but this is one feature that is much quicker in LSP, really the last feature that is better. Thank you.

[ngs428]

One other thought, possibly implement a staged approach to the artificial delay. If spamming is a concern, keep the delay for bulk searches over say 100 files. Anything less than 100 would have a much shorter or no delay. Small bulk searches would not have a spamming impact.

[jacquesh]

Why does it matter if bulk search takes a while? If you want to see lyrics for a track urgently in fb2k then you could just play the track and it would do the same thing bulk search would do anyway.

Can you elaborate a little on exactly when & why you're using bulk search? I might consider having a delay that starts small and ramps up if your query is large or something.

[ngs428]

Hi @jacquesh, thanks for the reply on this one. This is what I use for a process and when I would use bulk search.

I, and I assume others, mainly process new additions to my collection CD by CD, so 10 to maybe 60 tacks at a time depending on the release. I rip a CD I then feed it into MusicBrainz Picard to properly ID and tag it. After that I will run them through Foobar to add lyrics. So, I load the album in Foobar, select all tracks, right click, select download lyrics (using LSP 3). An album of 12 tracks, as I noted will take less thank 5 seconds to find matches and save alongside the FLAC files. As I noted in my other feature enhancement #401, I can see in the LSP window what all downloaded (source and timestamped or not), so I know if they are all good to go (source musixmatch and timestamped is what I look for) or if I need to review one and manually search.

This process takes maybe 20 seconds at most, then clean up any ones that didn't match by using the manual search I mentioned. Then on to the next disc that is probably done ripping. I've found over the years that processing lyrics in true bulk (say 100+ at a time) will just lead to a bit of a mess and cleanup. Smaller chunks and good into on what was downloaded is key to keeping accurate lyrics for each file and a collection that does not leave you scratching your head when you try to view lyrics (and they are not right or maybe way off timestamp wise).

Thanks for considering a varying delay based on how large the query is. I really think having bulk being able to process faster for small queries will be the best of both worlds. It won't promote spamming and it will still get users lyrics in a timely manner.

Thanks again for your work on this component! I really do think it is the best one out there, so I hope by giving this feedback as a seasoned user you will see what improvements can be made. Thank you.

[ngs428]

Hi @jacquesh one thing to keep in mind with the Basic Usage Metrics being collected (I signed up) is that some features may not get much use, for example this bulk search, simply because the user may not think they work well. I can use manual search faster than using the bulk search. So rather than seeing little usage of the feature and not focusing work on it, maybe it needs to be improved to increase usage. Just a thought. Have a good one and thanks for your work on this!