search

spotipy-dev / spotipy

A light weight Python library for the Spotify Web API
http://spotipy.readthedocs.org
MIT License
5.01k stars 956 forks source link

Spotipy user_playlists() function not fetching all user playlists #990

Closed msaini888 closed 1 year ago

msaini888 commented 1 year ago

Describe the bug Upon invoking the function with valid parameters , only 2 playlists out of total 3 playlists are fetched by the function call.

Your code import pandas as pd import spotipy from spotipy.oauth2 import SpotifyClientCredentials

Spotify API credentials

client_id = '' client_secret = '' username = ''

Create a dataframe to store information of my playlists

my_playlists = pd.DataFrame(columns=["id", "spotify_id", "list_name"])

Getting playlist info from Spotify

client_credentials_manager = SpotifyClientCredentials(client_id=client_id, client_secret=client_secret) sp = spotipy.Spotify(client_credentials_manager=client_credentials_manager)

playlists = [] results = sp.user_playlists(username) playlists.extend(results['items']) while results['next']: results = sp.next(results) playlists.extend(results['items'])

Extracting playlist information

for playlist in playlists: playlist_id = playlist['id'] spotify_id = playlist['uri'] list_name = playlist['name'] my_playlists = pd.concat([my_playlists, pd.DataFrame({"id": [playlist_id], "spotify_id": [spotify_id], "list_name": [list_name]})], ignore_index=True)

Creating dataframes to store song and feature information

my_songs = pd.DataFrame(columns=["list_id", "song_id", "song_name", "artist", "popularity", "release_date"]) my_features = pd.DataFrame(columns=["song_id", "energy", "liveness", "tempo", "speechiness", "acousticness", "instrumentalness", "danceability", "duration_ms", "loudness", "valence", "mode", "key"])

Getting song info and features from each playlist

for list_id in my_playlists["spotify_id"]: songs = [] playlist_tracks = sp.playlist_tracks(list_id, fields=None, limit=100, offset=0, market=None) songs += playlist_tracks['items'] while playlist_tracks['next']: playlist_tracks = sp.next(playlist_tracks) songs += playlist_tracks['items'] for song in songs: if song['track'] is None or song['track']['id'] is None: continue

    row = {
        "list_name" : list_name,
        "list_id": list_id,
        "song_id": song['track']['id'],
        "song_name": song['track']['name'],
        "artist": song['track']['artists'][0]['name'],
        "popularity": song['track']['popularity'],
        "release_date": song['track']['album']['release_date']
    }
    my_songs = pd.concat([my_songs, pd.DataFrame(row, index=[0])], ignore_index=True)

    features = sp.audio_features(tracks=[song['track']['id']])[0]
    row = {
         "song_id": song['track']['id'],
        "energy": features['energy'],
        "liveness": features['liveness'],
        "tempo": features['tempo'],
        "speechiness": features['speechiness'],
        "acousticness": features['acousticness'],
        "instrumentalness": features['instrumentalness'],
        "danceability": features['danceability'],
        "duration_ms": features['duration_ms'],
        "loudness": features['loudness'],
        "valence": features['valence'],
        "mode": features['mode'],
        "key": features['key']
    }
    my_features = pd.concat([my_features, pd.DataFrame(row, index=[0])], ignore_index=True)

Merging song and feature dataframes

list_song_feature = pd.merge(my_songs, my_features, left_on='song_id', right_on='song_id', how='left')

Exporting to CSV file

list_song_feature.to_csv('playlist_New.csv', index=False)

Expected behavior All 3 playlists should be fetched

Output {'href': 'https://api.spotify.com/v1/users/a07wqhmlqmvs0lfjbckmgiizu/playlists?offset=0&limit=50', 'items': [{'collaborative': False, 'description': 'Spaced Out', 'external_urls': {'spotify': 'https://open.spotify.com/playlist/7nAScwBKXcvE6V2oe6KQkb'}, 'href': 'https://api.spotify.com/v1/playlists/7nAScwBKXcvE6V2oe6KQkb', 'id': '7nAScwBKXcvE6V2oe6KQkb', 'images': [{'height': None, 'url': 'https://i.scdn.co/image/ab67706c0000bebb371b2e04e7f06eceac78a712', 'width': None}], 'name': 'My Playlist #8', 'owner': {'display_name': 'Mohit', 'external_urls': {'spotify': 'https://open.spotify.com/user/a07wqhmlqmvs0lfjbckmgiizu'}, 'href': 'https://api.spotify.com/v1/users/a07wqhmlqmvs0lfjbckmgiizu', 'id': 'a07wqhmlqmvs0lfjbckmgiizu', 'type': 'user', 'uri': 'spotify:user:a07wqhmlqmvs0lfjbckmgiizu'}, 'primary_color': None, 'public': True, 'snapshot_id': 'MjcwLDA1ZDM4YjRmNGIxZjA1Y2VkYjg2NDg1YjMzMmNlZWI2YmQwNjAzNWI=', 'tracks': {'href': 'https://api.spotify.com/v1/playlists/7nAScwBKXcvE6V2oe6KQkb/tracks', 'total': 255}, 'type': 'playlist', 'uri': 'spotify:playlist:7nAScwBKXcvE6V2oe6KQkb'}, {'collaborative': False, 'description': '', 'external_urls': {'spotify': 'https://open.spotify.com/playlist/2DuQV74JaCaXdSHsPML0uk'}, 'href': 'https://api.spotify.com/v1/playlists/2DuQV74JaCaXdSHsPML0uk', 'id': '2DuQV74JaCaXdSHsPML0uk', 'images': [{'height': 640, 'url': 'https://mosaic.scdn.co/640/ab67616d0000b27342f0dc3759a3b3e855245d10ab67616d0000b2737da6d6787071a97593cfbceeab67616d0000b2739618d5229371308305efecf1ab67616d0000b273f3278e0953010fc31bf147be', 'width': 640}, {'height': 300, 'url': 'https://mosaic.scdn.co/300/ab67616d0000b27342f0dc3759a3b3e855245d10ab67616d0000b2737da6d6787071a97593cfbceeab67616d0000b2739618d5229371308305efecf1ab67616d0000b273f3278e0953010fc31bf147be', 'width': 300}, {'height': 60, 'url': 'https://mosaic.scdn.co/60/ab67616d0000b27342f0dc3759a3b3e855245d10ab67616d0000b2737da6d6787071a97593cfbceeab67616d0000b2739618d5229371308305efecf1ab67616d0000b273f3278e0953010fc31bf147be', 'width': 60}], 'name': 'Spring_Bees', 'owner': {'display_name': 'Mohit', 'external_urls': {'spotify': 'https://open.spotify.com/user/a07wqhmlqmvs0lfjbckmgiizu'}, 'href': 'https://api.spotify.com/v1/users/a07wqhmlqmvs0lfjbckmgiizu', 'id': 'a07wqhmlqmvs0lfjbckmgiizu', 'type': 'user', 'uri': 'spotify:user:a07wqhmlqmvs0lfjbckmgiizu'}, 'primary_color': None, 'public': True, 'snapshot_id': 'MTE4LGRjYTJhYTg0NzU3YWNmZmFiNjNlYmQyNzdiOGE4YTc5NTliNjUxZDc=', 'tracks': {'href': 'https://api.spotify.com/v1/playlists/2DuQV74JaCaXdSHsPML0uk/tracks', 'total': 89}, 'type': 'playlist', 'uri': 'spotify:playlist:2DuQV74JaCaXdSHsPML0uk'}, {'collaborative': False, 'description': 'Best Deep House Relax selection. perfect for beach bar, roadtrip & shuffle. Follow Us: Instagram | Youtube ', 'external_urls': {'spotify': 'https://open.spotify.com/playlist/7ayyz9GOZVAx8shl9v3glz'}, 'href': 'https://api.spotify.com/v1/playlists/7ayyz9GOZVAx8shl9v3glz', 'id': '7ayyz9GOZVAx8shl9v3glz', 'images': [{'height': None, 'url': 'https://i.scdn.co/image/ab67706c0000bebb4984f8c9a918156cf9179894', 'width': None}], 'name': 'Deep House Relax 2023', 'owner': {'display_name': 'I NAME IT', 'external_urls': {'spotify': 'https://open.spotify.com/user/uz2sz8sxhi3xlynuzmrd4de50'}, 'href': 'https://api.spotify.com/v1/users/uz2sz8sxhi3xlynuzmrd4de50', 'id': 'uz2sz8sxhi3xlynuzmrd4de50', 'type': 'user', 'uri': 'spotify:user:uz2sz8sxhi3xlynuzmrd4de50'}, 'primary_color': None, 'public': True, 'snapshot_id': 'NTI4NSwxYzFlM2M2MDFlYTBlNmNlY2YyNThmNWNkYjM3ZTYxZWVkMmQ2N2Yz', 'tracks': {'href': 'https://api.spotify.com/v1/playlists/7ayyz9GOZVAx8shl9v3glz/tracks', 'total': 191}, 'type': 'playlist', 'uri': 'spotify:playlist:7ayyz9GOZVAx8shl9v3glz'}, {'collaborative': False, 'description': '', 'external_urls': {'spotify': 'https://open.spotify.com/playlist/5hKdqyebLQolYVouto4YpV'}, 'href': 'https://api.spotify.com/v1/playlists/5hKdqyebLQolYVouto4YpV', 'id': '5hKdqyebLQolYVouto4YpV', 'images': [{'height': 640, 'url': 'https://i.scdn.co/image/ab67616d0000b2736243471b5428210a62749dd2', 'width': 640}], 'name': 'Dalai Lama – Inner World', 'owner': {'display_name': 'Mohit', 'external_urls': {'spotify': 'https://open.spotify.com/user/a07wqhmlqmvs0lfjbckmgiizu'}, 'href': 'https://api.spotify.com/v1/users/a07wqhmlqmvs0lfjbckmgiizu', 'id': 'a07wqhmlqmvs0lfjbckmgiizu', 'type': 'user', 'uri': 'spotify:user:a07wqhmlqmvs0lfjbckmgiizu'}, 'primary_color': None, 'public': True, 'snapshot_id': 'Miw1NjMxZWQ0OTBiOTZiY2VmZDAyNTg5ZTgzNzQxNDcwZTY0MThmM2U0', 'tracks': {'href': 'https://api.spotify.com/v1/playlists/5hKdqyebLQolYVouto4YpV/tracks', 'total': 11}, 'type': 'playlist', 'uri': 'spotify:playlist:5hKdqyebLQolYVouto4YpV'}, {'collaborative': False, 'description': '', 'external_urls': {'spotify': 'https://open.spotify.com/playlist/75nZMpdvZ8gktV9495TFdo'}, 'href': 'https://api.spotify.com/v1/playlists/75nZMpdvZ8gktV9495TFdo', 'id': '75nZMpdvZ8gktV9495TFdo', 'images': [{'height': 640, 'url': 'https://mosaic.scdn.co/640/ab67616d0000b273230f7cd527da5c11dcd24f92ab67616d0000b27342c87abe0beec1b6a3879d44ab67616d0000b2737e679a65ecebffce1d32b64fab67616d0000b273bcee8e2aa4ded86f18661153', 'width': 640}, {'height': 300, 'url': 'https://mosaic.scdn.co/300/ab67616d0000b273230f7cd527da5c11dcd24f92ab67616d0000b27342c87abe0beec1b6a3879d44ab67616d0000b2737e679a65ecebffce1d32b64fab67616d0000b273bcee8e2aa4ded86f18661153', 'width': 300}, {'height': 60, 'url': 'https://mosaic.scdn.co/60/ab67616d0000b273230f7cd527da5c11dcd24f92ab67616d0000b27342c87abe0beec1b6a3879d44ab67616d0000b2737e679a65ecebffce1d32b64fab67616d0000b273bcee8e2aa4ded86f18661153', 'width': 60}], 'name': 'Sanctuary Of Sounds', 'owner': {'display_name': 'Mohit', 'external_urls': {'spotify': 'https://open.spotify.com/user/a07wqhmlqmvs0lfjbckmgiizu'}, 'href': 'https://api.spotify.com/v1/users/a07wqhmlqmvs0lfjbckmgiizu', 'id': 'a07wqhmlqmvs0lfjbckmgiizu', 'type': 'user', 'uri': 'spotify:user:a07wqhmlqmvs0lfjbckmgiizu'}, 'primary_color': None, 'public': True, 'snapshot_id': 'MTksOWRiYWVkNGI0Nzg3ODI1ZGY1NzRkYmQ3ZDgyMGJhYWRkY2EzZWQ2NA==', 'tracks': {'href': 'https://api.spotify.com/v1/playlists/75nZMpdvZ8gktV9495TFdo/tracks', 'total': 18}, 'type': 'playlist', 'uri': 'spotify:playlist:75nZMpdvZ8gktV9495TFdo'}], 'limit': 50, 'next': None, 'offset': 0, 'previous': None, 'total': 5} <class 'pandas.core.frame.DataFrame'>

Environment:

dieser-niko commented 1 year ago

First of all, please make sure to use a codeblock like this:

```python3 your code ```

Looking at your profile, I noticed one thing. The missing playlist is a collaborative playlist. Obviously this doesn't make much sense as the playlist is still public. I think it's a bug in Spotifys API and not in spotipy. Maybe somebody in the Spotify Community can help you with this.

I found this post which seems to describe the same problem: https://community.spotify.com/t5/Spotify-for-Developers/Get-User-s-Playlists-not-returning-quot-Old-Collaborative/m-p/5482577

msaini888 commented 1 year ago

thanks for the reply @dieser-niko , yes you are correct the playlist which is not getting returned is a collaborative playlist but since its public so it should get returned by the API call. I have already raised an issue regarding the same at Spotify Community as well but haven't received any response from them yet.

I cross checked the issue on Spotify developer console as well https://developer.spotify.com/documentation/web-api/reference/get-list-users-playlists?user_id=waggalishis&limit=50&offset=0 and the issue persists there as well.

msaini888 commented 1 year ago

should I close this issue?

dieser-niko commented 1 year ago

If the issue is not caused by spotipy, then probably yes.

msaini888 commented 1 year ago

okay closing the issue as this is happening from the Spotify API endpoint.