jakerm2002 / spotify-playlist-diff

0 stars 0 forks source link

Optimization: for large playlists >100 tracks #3

Open jakerm2002 opened 1 year ago

jakerm2002 commented 1 year ago

For large playlists, insert tracks into database immediately as they are received from the api.

Currently, the program waits for all tracks to be received in batches of 100 from the Spotify API. as the tracks are received, they are pushed into an array. once the array is filled with all the tracks we need, then we loop through each element of the array and call insert one by one.

Instead, as we receive batches of 100 tracks from the api, as soon as we receive them, we should insert them into our database, then fetch the next 100 tracks.

In addition, we should INSERT in bulk rather than looping through the 100 tracks one-by-one and calling an insert on each.