My library has grown too large for GMB. What are my options now?

[MountainX]

Due to the growing size of my music library, GMB has become very slow in most operations.

I have thought about splitting my library into different parts. I could probably accomplish this by maintaining multiple different ~/.config/gmusicbrowser/gmbrc files, each with a different path to a segment of my music library. I think I know how to accomplish the initial split.

To actually use GMB this way, I would rename a saved copy to the original name "gmbrc", then start GMB with that file. To switch to another part of my library, I would have to close GMB, copy & rename my rc files (using a script), and start GMB with a different file. Obviously, that's very cumbersome. Worse is that this doesn't allow for merging those different gmbrc files back into a single one when a better solution comes along.

Can anyone suggest better ideas?

Master filters help speed some things up, but my gmbrc file is over 43 MB now. That file size seems to be slowing everything down.

Edit - as a test, I moved about 10,000 songs out of my library. Afterwards, the gmbrc file size was not reduced at all. I expected it to become smaller after the scan was completed.

[squentin]

Could you send me the gmbrc file ? I'd love to test it and see what is slow and if I could improve it. (send it compressed or as a link to squentin at gmail.com, note that the last.fm password is saved in it)

Removed songs are not immediately removed from the database, due to concerns about losing the metadata that are not saved in the file tag. They are only removed after 20 "sessions", 1 "session" being 1 day where you launched gmb, so 20 days if you use it everyday. I should add a way to force removing these data though, maybe a button like "purge metadata from the x songs no longer in the library" or something.

About splitting it, it's not very easy, as some saved options reference songs in the library. I'll think about it but I'd much rather make it work with the full library.

[MountainX]

Could you send me the gmbrc file ?

Yes, gladly. And thank you!

Before removing about 10k songs, I had over 150,000 songs. It would be wonderful if I can keep using GMB for double or triple that number of songs!

Removed songs are not immediately removed from the database, due to concerns about losing the metadata that are not saved in the file tag.

The ideal solution for me would be to force all the metadata to be written into tags in the files. That way, if I can add the songs back to GMB later, I'll have my metadata.

[squentin]

Thanks for the file.

I think the biggest problem, assuming you are using the beta version using gtk3, is that that the songlist widget is very slow with filling and emptying long lists. The good news is that you can use the songtree widget for a nearly identical result. The layout you are using in the gmbrc you sent is Playlist that uses a PlayList widget, that widget is basically a songlist widget with some default options, but it can be made to use a songlist widget instead by adding the option songtree=1. Not sure if you need help editing the layout, basically search for the layout in layouts/main.layout, search for the layout that begins with [Playlist], and in it search for PlayList and replace is with PlayList(songtree=1), ask me if you need more help editing the layout. If the songtree widget groups songs together, you can just remove all the grouping to get mostly the same as a songlist widget by right clicking on the headers and select "Edit grouping..."

Other than that, the worst I've seen is a 2~3 seconds delay when selecting a filter with a lot of songs, obviously that might depend on your computer, maybe a bit longer when the filter is not yet cached. I think the PlayList widget is a bit slower because it has to mirror the changes to the playlist, so you could use a different layout unless you really like that behavior.

Obviously I don't have any of the cover pictures, so that might affect the speed too, though I think all uncached cover loading is delayed.

Can you tell me specifically what are the slowest/most annoying actions you experience ?

PS: By the way I noticed that the option to ignore articles when sorting messes up the A-Z menus that you get when clicking on the artist name (or album name with lots of other albums from same artist). I'll have to fix that.

[MountainX]

assuming you are using the beta version using gtk3

Yes, I am.

Can you tell me specifically what are the slowest/most annoying actions you experience ?

It is the Song Properties dialog. After I crossed about 100,000 songs, this dialog started having a slower response. In the current example, I have 10 songs (one album) in the Playlist. As I listen to each one, I open the Song Properties dialog to set metadata. The Song Properties dialog takes a couple seconds to become responsive. Initially, I cannot click into any field to enter or edit any values.

Because I use a lot of metadata fields (BPM, moods, themes, labels, rating), I have found that the Song Properties dialog is the most efficient way to work with metadata. But now that it has this slow response, it creates an unpleasant experience. Unfortunately, I have that experience with almost every song I listen to because when I listen to a new song, I always want to update the metadata.

The Song Properties dialog has a similar slowness when I have selected multiple tracks. A large selection of hundreds or thousands of tracks is only marginally slower to become responsive after it is opened (compared to a single song), but this multi-selection delay was also not present when my library was smaller.

you can use the songtree widget for a nearly identical result.

Thank you for the advice about using the songtree widget instead of the songlist widget. I will try that. However, the particular slowness is less annoying because I don't encounter it as frequently and I can easily work with smaller lists.

[squentin]

Oh ok, that slowness is very probably due to filling the completion models for artist/album entries. And it does seem to go away if you keep another song property dialog opened as the models are only destroyed when the last entry using it is destroyed. I'll think about how to fix this.

[MountainX]

Oh ok, that slowness is very probably due to filling the completion models for artist/album entries. And it does seem to go away if you keep another song property dialog opened as the models are only destroyed when the last entry using it is destroyed.

Yes, that does indeed work around the issue for me! Thank you. This actually helps me a lot, and I am OK to just leave one of those dialogs open.

in it search for PlayList and replace is with PlayList(songtree=1)

In ~/.config/gmusicbrowser/layouts/main.layout (and inside [Playlist]), I had this line:

HPfp_list     = _PlayList HPfp0

Replaced it with:

HPfp_list     = _PlayList(songtree=1) HPfp0

Upon starting GMB, it looks exactly the same, so I guess all is well. The grouping was set to none by default.

[MountainX]

I added my previously removed songs back to GMB. Library is over 150k songs now. With these two changes (using songtree and keeping one the Song Properties dialog open), GMB is much more responsive. Very usable again. In fact, the songtree is shocking fast at rendering without any filters. I did not expect it could be so fast.

[MountainX]

Oh ok, that slowness is very probably due to filling the completion models for artist/album entries. And it does seem to go away if you keep another song property dialog opened as the models are only destroyed when the last entry using it is destroyed. I'll think about how to fix this.

Do you have any suggestions for changes I could make to the code locally to remedy this?

BTW, with the suggestions you gave in this thread, I am no longer experiencing the issue.

[squentin]

An easy "fix" is disabling the completion, you can do that by editing gmusicbrowser_songs.pm search for "edit_listall => 1," and replace the 1 with 0.

[MountainX]

An easy "fix" is disabling the completion, you can do that by editing gmusicbrowser_songs.pm search for "edit_listall => 1," and replace the 1 with 0.

Although this works, I'm going to revert that change. I don't want to disable the filling of the completions. I want to disable their disposal.

You mentioned they are "destroyed when the last entry using it is destroyed." I'd like to try change that so they are destroyed only when I close GMB. (I suppose that might mean the completion items are not updated while GMB is in use, but I'd like to try it.)

Can you suggest a code change I can try?

[squentin]

You mentioned they are "destroyed when the last entry using it is destroyed." I'd like to try change that so they are destroyed only when I close GMB. (I suppose that might mean the completion items are not updated while GMB is in use, but I'd like to try it.)

Can you suggest a code change I can try?

Well, you can remove the line "::weaken $ExistingStores{$field};" in gmusicbrowser_songs.pm, but I'm not sure it'll work well as if for example you add an artist, it will reset and rebuild the artist liststore (it could be optimized but I didn't bother), so it might create an annoying delay when you set an artist and an album. Let me know. I'll be busy for around 2 weeks, but I'll try to find a good solution after.

[MountainX]

Well, you can remove the line "::weaken $ExistingStores{$field};" in gmusicbrowser_songs.pm, but I'm not sure it'll work well as if for example you add an artist, it will reset and rebuild the artist liststore (it could be optimized but I didn't bother), so it might create an annoying delay when you set an artist and an album. Let me know. I'll be busy for around 2 weeks, but I'll try to find a good solution after.

I have been trying this suggestion. It works pretty well, actually.

Of the solutions I have tried, the best is using songtree and keeping one of the Song Properties dialogs open all the time. Other Song Properties dialogs I open have no delays at all.

The second best solution is your last suggestion. I have not noticed any problem adding an artist, but if you want to give me specific steps to test, I'll do that and report back.