moritz-weber / mucke

mucke - android music player
GNU General Public License v3.0
224 stars 10 forks source link

Complete rewrite of local_music_fetcher_impl including faster import and multithreading #130

Closed FriederHannenheim closed 1 year ago

FriederHannenheim commented 1 year ago

So, this has been brewing for the last few days. When I changed the import from on_audio_query to metadata_god, I was dissatisfied with the longer import times. I've been thinking about how to fix that and couldn't find a solution that didn't include rewriting all of local_music_fetcher_impl. A few days ago I finally bit the sour apple and started work on it. This pull request is the culmination of my work. Let's see the results:

I am testing this on an OnePlus 8T with 8 logical CPU cores. On the current master a full library import takes 2 minutes and 2 seconds. With this branch I almost cut that time in half with the import only taking 1 minute and 8 seconds.

The hardest thing to get working was multithreaded generation of the background colors for the albumArt. At first I did not plan to do this but after my initial rewrite I noticed that it had only improved the import time by 15 Seconds. The generation of albumArt background colors was and is the longest step. If you leave out the code generating the background colors the whole import is done in 20 seconds.

After I had realized this I set out to multithread this too to improve speeds. This is where I hit a roadblock. I could not find a way to update the old code to run on other threads. Certain image functions where only available on the main flutter thread. That meant there was no way to use palette_generator outside of the main thread.

I tried hours to get it working to no avail. I considered just ripping out background colors for albums and songs. But something inside of me would not let me. Those colored backgrounds on the CurrentlyPlayingPage just look so good.

So I set out to implement a function that would pick a suitable color myself. With a lot of trial and error, I believe I have ended up on a function that works pretty well.

So, testing this please look at the backgrounds of albums and songs and judge if the color picked it suitable.

moritz-weber commented 1 year ago

Hi Frieder, thanks for all the work you put into this. Unfortunately, I won't be able to build the app myself in the coming months. So it would be great if you could attach a build that I can check out.

Alternatively, I can take a look at the Actions and fix the build Action for you. This way, there would always be builds available. This might take a few days, though.

Additionally, I would like @Valotio to give this a try because he had issues with the last implementation of the scan.

FriederHannenheim commented 1 year ago

Here is a build you and @Valotio can try: https://drive.proton.me/urls/7E8HV5MVJ8#sBz4Z5jDRDW2

moritz-weber commented 1 year ago

This build works really well for me. For some covers, the color generation is not ideal in my opinion, but still looks good. Nothing is really off, and I am fine with merging these changes.

One thing that significantly improved usability in the #124 branch was the progress indicator. If you like, you can take a look at the two streams I added there: https://github.com/moritz-weber/mucke/compare/master...124-app-hangs-while-importing-large-library#diff-2d2c530c13f8185189020ad2995059f45aaa347aef284d317fcbc1c191bce6cdR4

If you don't have the time to add this to your implementation, we can also release a new version without that and add it later.

FriederHannenheim commented 1 year ago

I've added the progress indicator. This was a thing I was missing with the library import too and I'm happy to see it working now. The progress indicator will scan through twice. Once for scanning the metadata on the files and a second time for generating the colors. I would have liked for it to be just one long progress indicator but that is sadly impossible

FriederHannenheim commented 1 year ago

Forgot to attach the build. I fixed the workflow in my branch so you can download the build artifact here: https://github.com/FriederHannenheim/mucke/actions/runs/5930287204