Closed fmigneault closed 2 years ago
fmigneault
commented
3 years ago Stepping deeper, I reach the following part of the code: https://github.com/tombulled/python-youtube-music/blob/048fac906c8aa6ab52ec3d39c2715503226a7543/ytm/parsers/album.py#L37-L46
The raised AssertionError is due to empty raw_mutations.
tombulled
commented
3 years ago Hi @fmigneault, thanks for raising this issue and helping to identify the cause.
It appears that YouTube have changed the API response, therefore a new parser will need to be created to get this working again. Unfortunately I am no longer actively maintaining this library, as I have focused my efforts on this libraries successor, innertube.
Here's a head start for either yourself, or anyone else interested in creating an updated parser (I may be able to free up some time in the future to help out):
import ytm
import pprint
base = ytm.BaseYouTubeMusic()
dl = ytm.YouTubeMusicDL()
album_playlist_id: str = 'OLAK5uy_mBL38OIl1pSnLmoPXS8JuMTku9ojLc3Yg'
page = base.page_playlist(album_playlist_id)
album_browse_id: str = ytm.utils.get \
(
page,
'INITIAL_ENDPOINT',
'browseEndpoint',
'browseId',
)
data = base.browse(album_browse_id)
album_title = ytm.utils.get \
(
data,
'header',
'musicDetailHeaderRenderer',
'title',
'runs',
0,
'text',
)
music_shelf_contents = ytm.utils.get \
(
data,
'contents',
'singleColumnBrowseResultsRenderer',
'tabs',
0,
'tabRenderer',
'content',
'sectionListRenderer',
'contents',
0,
'musicShelfRenderer',
'contents',
default = (),
)
tracks: list = []
for music_shelf_item in music_shelf_contents:
music_shelf_item = ytm.utils.first(music_shelf_item)
track_title = ytm.utils.get \
(
music_shelf_item,
'flexColumns',
0,
'musicResponsiveListItemFlexColumnRenderer',
'text',
'runs',
0,
'text',
)
track_video_id = ytm.utils.get \
(
music_shelf_item,
'playlistItemData',
'videoId',
)
tracks.append \
(
dict \
(
title = track_title,
video_id = track_video_id,
)
)
album = dict \
(
title = album_title,
tracks = tracks,
)
pprint.pprint(album)Which should output:
{'title': 'Lost in the Waves',
'tracks': [{'title': 'Lost in a Wave', 'video_id': 'uHSl0Zpw2pw'},
{'title': 'Rainfall', 'video_id': 'HVAIr-BhEZI'},
{'title': 'Silent', 'video_id': 'AIT1Rs7hiAg'},
{'title': 'Visage', 'video_id': 'NS7nEtuZwbM'},
{'title': 'Tired of It All', 'video_id': '-MqgkbxX9Dw'},
{'title': 'Say No Word', 'video_id': 'cXy1YodCDgQ'},
{'title': 'Always', 'video_id': '2r39hIbe4Xk'},
{'title': 'Shoreline', 'video_id': 'PvfQ4OkjIpE'},
{'title': 'Overrated', 'video_id': 'BfRecUhMIhM'},
{'title': 'Paralyzed', 'video_id': 'YNLZUinqksw'}]}On a side note, downloading has become much easier using innertube without requiring libraries such as youtube-dl. Here's a little example that outputs video data and streamable URLs:
import innertube
import pprint
client = innertube.InnerTube(innertube.Client.IOS_MUSIC)
data = client.player(video_id = 'uHSl0Zpw2pw')
pprint.pprint(data.videoDetails)
pprint.pprint(data.streamingData.adaptiveFormats)Hope that helps, and please let me know if you have any more issues :slightly_smiling_face:
fmigneault
commented
3 years ago @tombulled
Thanks for the reply. I will take a look at the proposed corrections to adjust parsing of the data in my spare time.
Regarding innertube, does it provide a similar method to download_album?
Would there be additional operations to create to obtain the files from the data returned by client.player?
tombulled
commented
3 years ago Unfortunately innertube is a lower-level library so you won't find convenience functions/methods such as download_album as I consider them out of scope for the library.
Here's a quick example for how you might go about downloading a single song using innertube, I'll try and add an example later on for downloading an entire album
import innertube # https://github.com/tombulled/innertube
# Third-party libraries
import addict # https://github.com/mewwts/addict
import requests # https://github.com/psf/requests
import slugify # https://github.com/un33k/python-slugify
import mutagen.mp4, mutagen.easymp4 # https://github.com/quodlibet/mutagen
# Standard libraries
import mimetypes
import pathlib
import shutil
# Add M4A to the `mimetypes` standard library
# as it's not included by default
mimetypes.types_map['.m4a'] = 'audio/mp4'
# LANDMVRKS - Lost in a Wave
video_id = 'AbAuyD3S818'
# Two clients are used
# - WEB_REMIX for video metadata
# - IOS_MUSIC for streamable URLs
# API responses differ between clients so they can
# be easily mixed and matched to get the data you need
web = innertube.InnerTube(innertube.Client.WEB_REMIX)
ios = innertube.InnerTube(innertube.Client.IOS_MUSIC)
# Dispatch `player` requests to the InnerTube API
web_player = web.player(video_id = video_id)
ios_player = ios.player(video_id = video_id)
details = web_player.videoDetails
# For simplicity we'll pick the last format,
# however you may desire to be more selective
format = ios_player.streamingData.adaptiveFormats[-1]
# Construct file name
path = pathlib.Path \
(
'{name}{extension}'.format \
(
name = slugify.slugify(details.title),
extension = mimetypes.guess_extension(format.mimeType.split(';')[0]),
),
)
# Download the audio file
with requests.get(format.url, stream = True) as response:
with path.open('wb') as file:
shutil.copyfileobj(response.raw, file)
# Download the album art and store its contents in memory
cover_image = requests.get(details.thumbnail.thumbnails[-1].url).content
# Create MP4 metadata tag containers
tags_easymp4 = mutagen.easymp4.EasyMP4()
tags_mp4 = mutagen.mp4.MP4()
# Add tags compatible with EasyMP4
tags_easymp4.update \
(
dict \
(
title = details.title,
artist = details.author,
album = 'Album Name Here', # Could just be omitted entirely if unknown
albumartist = details.author,
discnumber = str(1),
tracknumber = str(1),
)
)
# Add tags incompatible with EasyMP4
tags_mp4.update \
(
dict \
(
covr = \
(
mutagen.mp4.MP4Cover \
(
data = cover_image,
imageformat = mutagen.mp4.AtomDataType.JPEG,
),
),
),
)
# Write metadata tags
tags_easymp4.save(path)
tags_mp4.save(path)Here's an example to download an album/playlist:
import innertube # https://github.com/tombulled/innertube
# Third-party libraries
import addict # https://github.com/mewwts/addict
import furl # https://github.com/gruns/furl
import requests # https://github.com/psf/requests
import rich.console # https://github.com/willmcgugan/rich
import slugify # https://github.com/un33k/python-slugify
import mutagen.mp4, mutagen.easymp4 # https://github.com/quodlibet/mutagen
# Standard libraries
import mimetypes
import pathlib
import shutil
# Initialise a rich console for status updates
console = rich.console.Console()
# Add M4A to the `mimetypes` standard library
# as it's not included by default
mimetypes.types_map['.m4a'] = 'audio/mp4'
# InnerTube API client imitating iOS device
client = innertube.InnerTube(innertube.Client.IOS_MUSIC)
# URL to YouTube Music playlist
url = 'https://music.youtube.com/playlist?list=OLAK5uy_mBL38OIl1pSnLmoPXS8JuMTku9ojLc3Yg'
# Extract query parameters from the URL
params = addict.Dict(** furl.furl(url).query.params)
# Get the queue of tracks from the playlist
queue = client.music_get_queue(playlist_id = params.list)
# Extract the tracks from the queue response
tracks = \
[
item.content.playlistPanelVideoRenderer
for item in queue.queueDatas
]
with console.status("[bold green]Downloading tracks...") as status:
for track in tracks:
# Dispatch a `player` request to fetch streamable URLs
player = client.player(video_id = track.videoId)
# Extract video details
details = player.videoDetails
# For simplicity we'll pick the last format,
# however you may desire to be more selective
format = player.streamingData.adaptiveFormats[-1]
# Construct file name
path = pathlib.Path \
(
'{name}{extension}'.format \
(
name = slugify.slugify(details.title),
extension = mimetypes.guess_extension(format.mimeType.split(';')[0]),
),
)
# Download the audio file
with requests.get(format.url, stream = True) as response:
with path.open('wb') as file:
shutil.copyfileobj(response.raw, file)
# Download the album art and store its contents in memory
cover_image = requests.get(details.thumbnail.thumbnails[-1].url).content
# Create MP4 metadata tag containers
tags_easymp4 = mutagen.easymp4.EasyMP4()
tags_mp4 = mutagen.mp4.MP4()
# Tags compatible with EasyMP4
tags_easymp4.update \
(
dict \
(
title = details.title,
artist = details.author,
albumartist = details.author,
discnumber = str(1),
tracknumber = str(1),
)
)
# Tags incompatible with EasyMP4
tags_mp4.update \
(
dict \
(
covr = \
(
mutagen.mp4.MP4Cover \
(
data = cover_image,
imageformat = mutagen.mp4.AtomDataType.JPEG,
),
),
),
)
# Write metadata tags
tags_easymp4.save(path)
tags_mp4.save(path)
# Make a status update logging successful download of the file
console.log(f'Downloaded {details.title!r} -> {path.absolute()!s}')
I am using this package's
YouTubeMusicDLclass to obtain downloaded files from an album YTM link, with extended features in my project https://github.com/fmigneault/aiu.When I use the link https://music.youtube.com/playlist?list=OLAK5uy_mBL38OIl1pSnLmoPXS8JuMTku9ojLc3Yg, I get the following error traceback:
My project simply forwards the link to
YouTubeMusicDLin this case. There is not really any other step other than callingYouTubeMusicDL.download_albumdirectly with the album ID extracted from the link. I have managed to extract the result of thefunc(*args, **kwargs)step as presented below. For some reason, anAssertionErroris raised regardingalbum()which I cannot identify the cause.I have been able to use the exact same procedure with many other YTM links, but this one specifically (I have not yet been able to reproduce with others) causes this problem.