Podcast Topics to Spotify Playlists

[karl-cardenas-coding]

Hi Changelog friends. I was listening to Ship It episode 80, and I've been thinking about what @jerodsanto said about how he wishes he could create Spotify playlists (automatically) for all the podcast topics. I'd love to take a stab at addressing the challenge but wanted to make sure I understood the challenge correctly. Below are some questions that come to mind.

PS: Thanks for all the amazing work you all do ❤️

Questions

• There are currently 100+ topics and growing. Is the wish to have Spotify playlists have a 1:1 ratio with topics?
• I assume the RSS feed is the best source of truth, as I can pull metadata from it. Is this a correct assumption?

[jerodsanto]

This would be so cool, Karl! Thanks for showing interest 👏

To answer your immediate questions:

1. I would say "yes", but with a slight modification: Each topic that has published episodes associated to it would become a playlist on Spotify. So not all of them, but many.
2. This one depends on how you are thinking of building this feature. If it's part and parcel of our core Elixir application, then the source of truth is the database (via the topics and episodes tables). If, however, your plan is to build some standalone thing that watches our feeds and updates Spotify accordingly, then yes the RSS feed is the best source of truth. However, I think the former approach is the better one...

Please follow up and I can clarify as we go, thanks! 💚

[karl-cardenas-coding]

Thanks for the clarification @jerodsanto. This helps me with the design aspect. I was thinking of creating a more general tool, something along the lines of a CLI that I publish as a binary and a container. That way you can control how frequently you want to run the logic. The logic could be executed as a cron job or something you add to your deployment process (maybe the transcript repo??). I rather the consumer decide how to run this without complicating an existing codebase. Just kinda what I am leaning towards 😅

Edit: Using the RSS feed as the source won't work as the categories don't map back to the topic list. Otherwise we end up with over 14k playlists vs 230+ 🤣

The way I see it, we have two options, create a solution that queries the DB directly OR use the topics page as the source of truth. The latter feels brittle as any UI changes would require logic update.

[karl-cardenas-coding]

What are the chances an API exists that exposes the topics? Probably slim, I assume.

[jerodsanto]

We do not have a public API. However, if/when we get the topic-based RSS feeds shipped you'd be able to pull appropriate podcast episodes from those feeds.

Using something like that, your generic tool's description would essentially be:

Give this tool an RSS feed and a name and it will maintain a playlist for you inside Spotify.

The other challenge will be mapping the episodes in podcaster feeds to the appropriate episode inside Spotify. I do not believe they use/retain our GUIDs, which means you'd be stuck doing an episode name match maybe? Also quite brittle...

That fact is worth confirming, though, because if I'm wrong this process is a whole lot easier...

[karl-cardenas-coding]

@jerodsanto, you are correct, about the Spotify lookup. We would have to query the Spotify API to get all the episodes from the show(s) and their respective Spotify ID, followed by a subsequent request to add the episode to the playlist. We would also have to ensure the playlist does not contain the episode. The latter part becomes tricky without persistent storage or a location where the state can be maintained. Otherwise, there would be a lot of API calls to Spotify. The topic-based RSS feed would make this a more manageable challenge, but that doesn't address the playlist "state" challenge.

The other more realistic option is to query the DB and retrieve each topic's topics and episode names. That would reduce the need to scrape the data from the website or through an RSS feed. We could then create a table in the DB that stores all the playlists and their episodes. This "playlists" table would be the source of truth for all playlists. At that point, the only thing left is basic Spotify API calls, such as acquiring episode id and adding the episode to the playlist.

I can probably mock something up if there is still interest. But it would help if I could replicate something similar to the real DB schema and data.

[jerodsanto]

@karl-cardenas-coding with that much effort required, I think putting some thought/work into an overall "playlists" feature for the site would make the most sense. Then a sub-feature of the playlists feature would be the ability to toggle a one-way sync to Spotify...

[gerhard]

Do we still want to keep this open? FWIW, this is just around the corner:

• https://github.com/thechangelog/changelog.com/discussions/440

[karl-cardenas-coding]

@gerhard I can listen to the episode and leave a comment once it released.

[jerodsanto]

Dang, we never got around to discussing this topic on Kaizen 9! 😭

[adamstac]

Hopefully we'll Kaizen again soon enough!