Retrieving the full tracks for an album

[toothbrush]

Hi there – another n00b question from me 😅

I'm attempting to write a function that takes an album URI, and returns the full-hydrated list of Track objects (not the "simplified" ones i'm able to get with albumTracks()). I'm finding it quite difficult / cumbersome, but that's probably because i don't know how to use Combine properly. I've cobbled something together based on the example in https://github.com/Peter-Schorn/SpotifyAPIExamples/blob/main/SpotifyAPIExamples/main.swift and the Wiki entry about pagination.

Unfortunately, it's still extremely verbose. I've put my function in a Gist: https://gist.github.com/toothbrush/a34b38506dddda07e1b40a17adce17ce

Is there a nicer way of achieving this task? I could just use albumTracks(), but that just gives me "simplified" Track objects.

Since i mostly just want the Track objects to at least know their album name and album URI, i thought of trying something like this – i hope my pseudocode makes sense.

// ... get list of simplified Tracks
if let tracks = fullAlbum?.tracks {
  for t in tracks {
    t.album = fullAlbum
  }
}

But of course Track has immutable fields. Or perhaps there's a way to let the tracks hydrate on a background thread and update once they're ready?

Thank you for your patience!

[Peter-Schorn]

Are you trying to get all of the tracks in an album, or the full versions of the tracks, or both? Also, do you need to retrieve the full album object, or just the tracks? And is it ok if you wait for all tracks to be retrieved before processing them further? Or do you need to process each page of tracks as it is received (e.g., you want to add each page to a table as it is received)?

[Peter-Schorn]

Here's an extension on SpotifyAPI that returns the full versions of all tracks in an album, one page at a time. It's unfortunate that there is no efficient way to retrieve this data. You must first retrieve the simplified versions of the tracks and then pass them into SpotifyAPI.tracks(_:market:) to get the full versions:

extension SpotifyAPI {

    /**
     Retrieves the *full* versions of all the tracks in an album.

     - Parameters:
       - album: The URI for an album
       - market: *Optional*. An [ISO 3166-1 alpha-2 country code][2] or the
             string "from_token". Provide this parameter if you want to apply
             [Track Relinking][1].
     - Returns: A publisher that publishes an array of the full versions of
           track objects, **one page at a time**. Each page will contain up to
           50 tracks.

     [3]: https://developer.spotify.com/documentation/general/guides/track-relinking-guide/
     */
    func albumFullTracks(
        _ album: SpotifyURIConvertible,
        market: String? = nil
    ) -> AnyPublisher<[Track], Error> {

        self.albumTracks(
            album,
            market: market,
            // `SpotifyAPI.tracks(_:market:)` (used below) accepts a maximum of 50
            // tracks
            limit: 50
        )
        .extendPages(self)
        // extract the URIs of the tracks from each page
        .map { tracksPage in tracksPage.items.compactMap(\.uri) }
        .flatMap { trackURIs -> AnyPublisher<[Track?], Error> in
            // accepts a maximum of 50 tracks
            return self.tracks(trackURIs)
        }
        // remove the `nil` items from the array of tracks
        .map { $0.compactMap { $0 } }
        .eraseToAnyPublisher()

    }

}

// example usage:

let spotifyAPI: SpotifyAPI = ...
let album: SpotifyURIConvertible = ...

var cancellables: Set<AnyCancellable> = []
let dispatchGroup = DispatchGroup()

dispatchGroup.enter()
spotifyAPI.albumFullTracks(album)
    .sink(
        receiveCompletion: { completion in
            print("\ncompletion: \(completion)")
            dispatchGroup.leave()
        },
        receiveValue: { tracks in
            print("\n--- received \(tracks.count) tracks ---")
            for track in tracks {
                print(track.name)
            }
        }
    )
    .store(in: &cancellables)
dispatchGroup.wait()

[Peter-Schorn]

If the order that the tracks are received in doesn't matter then there's a faster method.

[toothbrush]

Hey, thanks for the response! Yeah, i'd prefer to have the full Track, i don't mind much about the album. I guess in that case it makes sense that albumTracks() followed by a "hydrate" step would be best...

As for order, in this case i think i don't mind, because i'm adding them to an array which is used by an Array Controller and takes care of sorting them and presenting them in NSTableView – so i'd be keen to try the faster method and compare to what you've suggested for the order-preserving method above.

Thank you for your help!

[toothbrush]

I wonder if there would be a way to do the following:

1. use albumTracks() which is fast, to get the full list of "simplified" Tracks
2. add them to my array controller, but in the background: a. Tell each of them something like track.elaborate() which will reach out to the Spotify API to populate missing fields? Or is this impossible 😅

[Peter-Schorn]

Tell each of them something like track.elaborate()

You definitely have an eccentric vocabulary 😂. This is not possible because all of the members of Track are immutable. In order to achieve this behavior, you must store the Track object as a mutable property inside a wrapper type (it looks like you have a type called RBSpotifySongTableRow which could do this). First you store the simplified version in the wrapper object (RBSpotifySongTableRow.track = simplifiedTrack), and then when you retrieve the full version of the track you assign this version to the wrapper type (RBSpotifySongTableRow.track = fullTrack).

so i'd be keen to try the faster method and compare to what you've suggested for the order-preserving method above.

In order to use the faster method, replace .extendPages(self) with .extendPagesConcurrently(self) in the SpotifyAPI.albumFullTracks(_:market:) method I defined above. This will cause the pages to be returned in an unpredictable order.

The speed improvement is proportional to the speed of each network request and the number of pages that you retrieve. If you retrieve two pages or less, then there is no difference in speed.

[toothbrush]

You definitely have an eccentric vocabulary 😂

😛

First you store the simplified version in the wrapper object (RBSpotifySongTableRow.track = simplifiedTrack), and then when you retrieve the full version of the track you assign this version to the wrapper type (RBSpotifySongTableRow.track = fullTrack).

You guessed correctly – i have such a wrapper. That sounds like a reasonable approach, thanks! I'd then probably have to call something like reloadData() on the NSTableView to get it to actually refresh e.g. the album name once that's populated, but that's doable.

[toothbrush]

The speed improvement is proportional to the speed of each network request and the number of pages that you retrieve. If you retrieve two pages or less, then there is no difference in speed.

Ah that's good to know. If i'm doing this for albums, and the vast majority of albums have <50 tracks, i might just keep the "slow" method. Thanks for all the input!

[Peter-Schorn]

If i'm doing this for albums, and the vast majority of albums have <50 tracks, i might just keep the "slow" method.

Because there is only a difference in speed if you retrieve more than two pages, this translates to an improvement in speed if there is more than 100 tracks (50 per page * 2 pages). As you alluded to, it should be extremely rare for an album to have more than 100 tracks.

[Peter-Schorn]

Here's an example that retrieves the simplified tracks and stores them in an array of RBSpotifySongTableRow and then updates this array with the full tracks as they are received.

let spotifyAPI: SpotifyAPI = ...
let album: SpotifyURIConvertible = ...

var cancellables: Set<AnyCancellable> = []

var tableRows: [RBSpotifySongTableRow] = []

spotifyAPI.albumTracks(
    album,
    // `SpotifyAPI.tracks(_:market:)` (used below) accepts a maximum of 50
    // tracks
    limit: 50
)
.extendPages(spotifyAPI)
.receive(on: RunLoop.main)
.flatMap { tracksPage -> AnyPublisher<[Track?], Error> in

    print("received page \(tracksPage.estimatedIndex)")

    // all of the tracks in the page
    let simplifiedTracks = tracksPage.items

    // create a new array of table rows from the page of simplified tracks
    let newTableRows = simplifiedTracks.map(
        RBSpotifySongTableRow.init(track:)
    )

    // append the new table rows to the full array
    tableRows.append(contentsOf: newTableRows)

    // retrieve just the URIs of the tracks
    let trackURIs = simplifiedTracks.compactMap(\.uri)

    // retrieve the full versions of the tracks
    return spotifyAPI.tracks(trackURIs)

}
.receive(on: RunLoop.main)
.sink(
    receiveCompletion: { completion in
        print("\ncompletion: \(completion)")
    },
    receiveValue: { fullTracks in
        // receive the full tracks
        print("\n--- received \(fullTracks.count) tracks ---")
        guard let firstTrack = fullTracks.first as? Track else {
            return
        }

        // find the index of the first track in this page in the table rows
        guard let firstTrackInPageIndex = tableRows.firstIndex(
            where: { tableRow in
                tableRow.track.uri == firstTrack.uri
            }
        ) else {
            print(
                "error: could not find index of \(firstTrack.name) in " +
                "table rows"
            )
            return
        }

        // replace the simplified versions of the tracks with the full
        // versions in the table rows
        for (index, fullTrack) in fullTracks.enumerated() {
            guard let fullTrack = fullTrack else {
                continue
            }
            let tableRowTrackIndex = firstTrackInPageIndex + index
            #if DEBUG
            let simplifiedTrack = tableRows[tableRowTrackIndex].track
            // assert that `simplifiedTrack` and `fullTrack` are in fact the
            // same
            assert(simplifiedTrack.uri == fullTrack.uri)
            #endif

            // replace the simplified track with the full track
            tableRows[tableRowTrackIndex].track = fullTrack
        }

    }
)
.store(in: &cancellables)

// only required in a command-line project, which I'm using
RunLoop.main.run()