sigma67 / ytmusicapi

Unofficial API for YouTube Music
https://ytmusicapi.readthedocs.io
MIT License
1.59k stars 182 forks source link

`KeyError: 'fixedColumns'` in specific circumstance when retrieving uploads #578

Closed apastel closed 1 month ago

apastel commented 2 months ago

It seems like Google made a minor change with regard to uploading songs that is affecting get_library_upload_songs()

Now when you upload a new song via the web page, after the song finishes processing, it may first appear in your library without a duration:

image

Then after about 10 seconds or so, when you refresh it will have the duration:

image

During the brief period of time when there is not yet a duration, get_library_upload_songs() throws a KeyError for the fixedColumns attribute, since that entire attribute will be missing:

Traceback (most recent call last):
  File "/home/apastel/dev/ytmusic-deleter/test.py", line 5, in <module>
    response = yt_auth.get_library_upload_songs()
  File "/home/apastel/dev/ytmusic-deleter/.venv/lib/python3.10/site-packages/ytmusicapi/mixins/uploads.py", line 59, in get_library_upload_songs
    songs = parse_uploaded_items(results["contents"])
  File "/home/apastel/dev/ytmusic-deleter/.venv/lib/python3.10/site-packages/ytmusicapi/parsers/uploads.py", line 33, in parse_uploaded_items
    duration = get_fixed_column_item(data, 0)["text"]["runs"][0]["text"]
  File "/home/apastel/dev/ytmusic-deleter/.venv/lib/python3.10/site-packages/ytmusicapi/parsers/_utils.py", line 48, in get_fixed_column_item
    "text" not in item["fixedColumns"][index]["musicResponsiveListItemFixedColumnRenderer"]
KeyError: 'fixedColumns'

I feel like this is a new change from Google because I'm suddenly experiencing it in unit tests and one user of my app reported it today. It probably reduces the upload processing time on their part by letting the duration get filled in lazily afterwards. I was able to reproduce it outside my app in just a basic test file, and I ran this repeatedly after uploading a song.

from ytmusicapi import YTMusic

yt_auth = YTMusic("./tests/resources/oauth.json")
response = yt_auth.get_library_upload_songs()
print(response)

So it probably amounts to an attribute check and maybe we just don't include the duration in the results (or None) if that happens?

without_fixedColumns.json

apastel commented 1 month ago

Google is definitely messing with this currently. Now instead of not including the fixedColumns object in the response, it's now included but the duration is set to ' ' which breaks the parser:

.venv/lib/python3.12/site-packages/ytmusicapi/mixins/uploads.py:59: in get_library_upload_songs
    songs = parse_uploaded_items(results["contents"])
.venv/lib/python3.12/site-packages/ytmusicapi/parsers/uploads.py:39: in parse_uploaded_items
    "duration_seconds": parse_duration(duration),
.venv/lib/python3.12/site-packages/ytmusicapi/parsers/_utils.py:67: in parse_duration
    seconds = sum(multiplier * int(time) for multiplier, time in mapped_increments)
E   ValueError: invalid literal for int() with base 10: ' '

And although I said that this only happens right after uploading a new song, I had it happen on a user that had about 10k uploaded songs that had been sitting there for years, so I guess it's possible for any uploaded song to have a blank duration.

If I have time I will contribute a PR that will probably look something like this to cover both cases if there are no fixedColums and if the duration is just whitespace or empty string:

def parse_uploaded_items(results):
...
...
        duration = None
        if "fixedColumns" in data:
            duration = get_fixed_column_item(data, 0)["text"]["runs"][0]["text"]
def parse_duration(duration):
    if not duration or not duration.strip():
        return None
    mapped_increments = zip([1, 60, 3600], reversed(duration.split(":")))
    seconds = sum(multiplier * int(time) for multiplier, time in mapped_increments)
    return seconds