Currently all tracks are saved as one giant object inside indexedDB every time one of the tracks changes, that makes easy to work with data inside the app, but performance and especially first load degrades quickly with large amount of tracks (1000+). Main cause of that is slow indexedDB.
Loading data in chunks could help with first load, but then how do you sort or search incomplete data?
Even then it's still loading all of the users data from disk to the memory, despite app only showing small parts of it at any given moment.
The optimal solution and most radical one requiring full rearchitecture of the app would be to use SQL. I haven't tested this library, but it says it's way faster than raw IndexedDB when working with large amount of data and doesn't keep all of it in memory. Unfortunately there two big problems with it: listening for data changes, right now if delete the track it's deleted everywhere automatically, that wouldn't be the case anymore and more significant issue that it's 1MB minified package, as of this moment the main app bundle size is about 107kb minified, that's 10x times increase in size. With GZIP, it's 400kb vs 30kb so 13x.
Though the bonus of adopting that library would be that we can use full SQL language features so no need for manually keeping relations between track -> artist and so on. Searching and filtering would be done on separate thread also. The data loading would be as if working with fetch rest api. Since list's are virtualized data could be loaded dynamically as user scrolls.
This isn't a critical issue for now, just an exploration of how to improve things down the line.
Currently all tracks are saved as one giant object inside indexedDB every time one of the tracks changes, that makes easy to work with data inside the app, but performance and especially first load degrades quickly with large amount of tracks (1000+). Main cause of that is slow indexedDB.
https://github.com/minht11/local-music-pwa/blob/cf1a2f3a16266140d0e964b9952376957d039fe5/src/stores/entities/create-entities-store.ts#L298-L300 https://github.com/minht11/local-music-pwa/blob/cf1a2f3a16266140d0e964b9952376957d039fe5/src/stores/persist-stores.tsx#L45-L53
Loading data in chunks could help with first load, but then how do you sort or search incomplete data? Even then it's still loading all of the users data from disk to the memory, despite app only showing small parts of it at any given moment.
The optimal solution and most radical one requiring full rearchitecture of the app would be to use SQL. I haven't tested this library, but it says it's way faster than raw IndexedDB when working with large amount of data and doesn't keep all of it in memory. Unfortunately there two big problems with it: listening for data changes, right now if delete the track it's deleted everywhere automatically, that wouldn't be the case anymore and more significant issue that it's 1MB minified package, as of this moment the main app bundle size is about 107kb minified, that's 10x times increase in size. With GZIP, it's 400kb vs 30kb so 13x.
Though the bonus of adopting that library would be that we can use full SQL language features so no need for manually keeping relations between track -> artist and so on. Searching and filtering would be done on separate thread also. The data loading would be as if working with fetch rest api. Since list's are virtualized data could be loaded dynamically as user scrolls.
This isn't a critical issue for now, just an exploration of how to improve things down the line.