search

spotDL / spotify-downloader

Download your Spotify playlists and songs along with album art and metadata (from YouTube if a match is found).
https://spotdl.readthedocs.io/en/latest/
MIT License
17.16k stars 1.58k forks source link

TypeError: object of type 'NoneType' has no len() #1120

Closed old-df38257622 closed 3 years ago

old-df38257622 commented 3 years ago

Environment

Expected Behaviour:

Downloads playlist with no issue.

Actual Behaviour

After getting a good amount through the playlist, it seems to hang on Eminem - Like Toy Soldiers.

Steps to Reproduce

  1. Install spotdl as shown above
  2. python3.8 -m spotdl https://open.spotify.com/playlist/0B050DzFrUxxqzcsyDP96h
  3. Wait

MOD EDITED: spotdl https://open.spotify.com/track/6CN3e26iQSj1N5lomh0mfO?si=_Ia2Nis1Tu2RqXQiG2OhwQ is the song that causes error.

Traceback

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 130, in <module>
    console_entry_point()
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 110, in console_entry_point
    songObjList = get_playlist_tracks(request)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/utils.py", line 82, in get_playlist_tracks
    song = SongObj.from_url(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/songObj.py", line 63, in from_url
    youtubeLink = SongObj.searchProvider(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 253, in search_and_get_best_match
    results = search_and_order_ytm_results(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 147, in search_and_order_ytm_results
    results = __query_and_simplify(get_ytm_search_query(songName, songArtists))
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 122, in __query_and_simplify
    return list(map(__map_result_to_song_data, searchResult))
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 95, in __map_result_to_song_data
    'length': __parse_duration(result['duration']),
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 76, in __parse_duration
    if len(duration) > 5:
TypeError: object of type 'NoneType' has no len()
Silverarmor commented 3 years ago

Can reproduce on Windows 10 Pro, Python 3.9.1, spotDL v3.3.0

Silverarmor commented 3 years ago

Also reproducible for Adele - Someone Like You spotdl https://open.spotify.com/track/3bNv3VuUOKgrf5hu3YcuRo?si=wTd-AfL8Q9SfgERGwJLhfg

(Reported by Death Trooper#9191 on Discord)

old-df38257622 commented 3 years ago

Good to know I'm not the only one having this issue.

Silverarmor commented 3 years ago

Labelled High Priority due to many users experiencing this issue (as reported on Discord also)

@spotDL/joiners - Could be a good first troubleshoot @spotDL/contributors - Assign yourself to the issue if you can help.

ghost commented 3 years ago

Look I don't know what will that effect, but it worked for me. I was having the same error. Here is a simple solution. Don't laugh.

I went to that provider.py file which was causing the problem ( you can see in the error stack ) Then, in that function I added that whole if...else block in a try block and then in the except... wait, let me copy and paste the code to make things easy.

old = ''
def __parse_duration(duration: str) -> float:
    global old
    try:
        if len(duration) > 5:
            old = duration
            padded = duration.rjust(8, '0')
            x = strptime(padded, '%H:%M:%S')
        elif len(duration) > 2:
            padded = duration.rjust(5, '0')
            x = strptime(padded, '%M:%S')
        else:
            x = strptime(duration, '%S')
    except:
        padded = old.rjust(8, '0')
        x = strptime(padded, '%H:%M:%S')

    return timedelta(hours=x.tm_hour, minutes=x.tm_min, seconds=x.tm_sec).total_seconds()

Now, that solves the problem! All songs get downloaded...

I don't know where will that give a problem, but if you face any, please let me know.

old-df38257622 commented 3 years ago

It looks like that solves the problem when applied to my local install. I'll go through and download my music, and let you know if I have any other issue - in the meantime, maybe open a PR?

aklajnert commented 3 years ago

@AayushSameerShah that's very similar to my solution in #1126, but your code is a bit weird. You don't need to define old as global, it'd be enough to just have it defined within the function instead of outside of it.

old-df38257622 commented 3 years ago

Although the fix gets the songs to all search correctly, a new problem pops up during download, though I'm not sure if this is related to the fix, and may be neither here nor there in this issue's scope:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 130, in <module>
    console_entry_point()
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 112, in console_entry_point
    downloader.download_multiple_songs(songObjList)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/download/downloader.py", line 91, in download_multiple_songs
    self._download_asynchronously(songObjList)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/download/downloader.py", line 367, in _download_asynchronously
    self.loop.run_until_complete(asyncio.gather(*tasks))
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/download/downloader.py", line 362, in _pool_download
    return await self.download_song(song_obj)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/download/downloader.py", line 172, in download_song
    youtubeHandler = YouTube(
  File "/home/letters/.local/lib/python3.8/site-packages/pytube/__main__.py", line 103, in __init__
    self.descramble()
  File "/home/letters/.local/lib/python3.8/site-packages/pytube/__main__.py", line 123, in descramble
    self.player_config_args = get_ytplayer_config(self.watch_html)[
  File "/home/letters/.local/lib/python3.8/site-packages/pytube/extract.py", line 205, in get_ytplayer_config
    raise RegexMatchError(
pytube.exceptions.RegexMatchError: get_ytplayer_config: could not find match for config_patterns

It makes it to 2% progress before this error pops up. I feel like this is either an issue with pytube or a completely seperate spotdl issue, but either way, I'd like a second opinion on if it belongs here.

Silverarmor commented 3 years ago

@keli5 pip install --no-cache-dir -U git+https://github.com/pytube/pytube Update ur pytube?

old-df38257622 commented 3 years ago

Doesn't seem to change anything, same version of pytube is installed, same error is raised.

Silverarmor commented 3 years ago

@keli5 can you confirm that you are running on the dev branch of spotDL? You can join our discord and chat there if it is easier also.

aklajnert commented 3 years ago

@keli5 - what's the URL you trying to download?

fenixnet-net commented 3 years ago

I've repro'd, but with some additional interesting tidbits.

To start: docker run -it --name spotdl debian:latest Inside the container: apt-get update ; apt-get install -y ffmpeg python3-pip Wait for almost a gig of stuff to download and install...

Finally: pip3 install https://codeload.github.com/spotDL/spotify-downloader/zip/dev

Now on to break things.

First, trying a single song:

Fetching Song...
Searching for: K/DA, Madison Beer, (G)I-DLE, Jaira Burns, League of Legends - POP/STARS
 95%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋       |ETA: 00:00, 0.02min/song
Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 10, in <module>
    sys.exit(console_entry_point())
  File "/usr/local/lib/python3.7/dist-packages/spotdl/__main__.py", line 96, in console_entry_point
    downloader.download_single_song(song)
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 74, in download_single_song
    self._download_asynchronously([songObj])
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 367, in _download_asynchronously
    self.loop.run_until_complete(asyncio.gather(*tasks))
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 362, in _pool_download
    return await self.download_song(song_obj)
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 251, in download_song
    audioFile = EasyID3(convertedFilePath)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/easyid3.py", line 169, in __init__
    self.load(filename)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 185, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 156, in wrapper
    return func(self, h, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/id3/_file.py", line 152, in load
    self._header = ID3Header(fileobj)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 185, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/id3/_tags.py", line 59, in __init__
    raise ID3NoHeaderError("%s: too small" % fn)
mutagen.id3._util.ID3NoHeaderError: KDA, Madison Beer, (G)I-DLE, Jaira Burns, League of Legends - POPSTARS.mp3: too small
Exception ignored in: <function tqdm.__del__ at 0x7fb78551b510>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1134, in __del__
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1281, in close
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1467, in display
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1137, in __repr__
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/progressHandlers.py", line 23, in format_dict
  File "/usr/local/lib/python3.7/dist-packages/tqdm/std.py", line 1424, in format_dict
TypeError: cannot unpack non-iterable NoneType object

Sadsauce. Perhaps an album?

Fetching Album...
Searching for: K/DA, (G)I-DLE, Wolftyla, Bea Miller, League of Legends - THE BADDEST
Searching for: K/DA, Madison Beer, (G)I-DLE, Lexie Liu, Jaira Burns, Seraphine, League of Legends - MORE
Searching for: K/DA, Madison Beer, Kim Petras, League of Legends - VILLAIN
Searching for: K/DA, Wolftyla, Bekuh Boom, Aluna, League of Legends - DRUM GO DUM
Searching for: K/DA, TWICE, Bekuh Boom, Annika Wells, League of Legends - I'LL SHOW YOU
 58%|████████████████████████████████████████████████████████████████████████████████████▌                                                             |ETA: 00:03, 0.03min/song
Traceback (most recent call last):
  File "/usr/local/bin/spotdl", line 10, in <module>
    sys.exit(console_entry_point())
  File "/usr/local/lib/python3.7/dist-packages/spotdl/__main__.py", line 106, in console_entry_point
    downloader.download_multiple_songs(songObjList)
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 91, in download_multiple_songs
    self._download_asynchronously(songObjList)
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 367, in _download_asynchronously
    self.loop.run_until_complete(asyncio.gather(*tasks))
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 362, in _pool_download
    return await self.download_song(song_obj)
  File "/usr/local/lib/python3.7/dist-packages/spotdl/download/downloader.py", line 251, in download_song
    audioFile = EasyID3(convertedFilePath)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/easyid3.py", line 169, in __init__
    self.load(filename)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 185, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 156, in wrapper
    return func(self, h, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/id3/_file.py", line 152, in load
    self._header = ID3Header(fileobj)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/_util.py", line 185, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/mutagen/id3/_tags.py", line 59, in __init__
    raise ID3NoHeaderError("%s: too small" % fn)
mutagen.id3._util.ID3NoHeaderError: KDA, (G)I-DLE, Wolftyla, Bea Miller, League of Legends - THE BADDEST.mp3: too small
 81%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋                            |ETA: 00:01, 0.03min/song

Alas. Interestingly this output also kills echo to terminal. I had to run a reset to get my command line echo back.

We DO have some artifacts left over. Let's have a look at those:

0   KDA, (G)I-DLE, Wolftyla, Bea Miller, League of Legends - THE BADDEST.mp3
0   KDA, Madison Beer, (G)I-DLE, Jaira Burns, League of Legends - POPSTARS.mp3
0   KDA, Madison Beer, (G)I-DLE, Lexie Liu, Jaira Burns, Seraphine, League of Legends - MORE.mp3
0   KDA, Madison Beer, Kim Petras, League of Legends - VILLAIN.mp3
8.0K    POPSTARS.spotdlTrackingFile
28K THE BADDEST.spotdlTrackingFile
19M Temp

du -h Temp/*
2.6M    Temp/KDA (G)I-DLE Wolftyla Bea Miller League of Legends - THE BADDEST.webm
3.0M    Temp/KDA Madison Beer (G)I-DLE Jaira Burns League of Legends - POPSTARS.webm
3.4M    Temp/KDA Madison Beer (G)I-DLE Lexie Liu Jaira Burns Seraphine League of Legends - MORE.webm
3.2M    Temp/KDA Madison Beer Kim Petras League of Legends - VILLAIN.webm
3.2M    Temp/KDA TWICE Bekuh Boom Annika Wells League of Legends - ILL SHOW YOU.webm
3.3M    Temp/KDA Wolftyla Bekuh Boom Aluna League of Legends - DRUM GO DUM.webm

Turns out VLC can handle those webm files just fine. Media stats show ~128kb/s, 48kHz 32-bit sample rate. Doing a transcode to mp3 using VLC succeeded.

Looks like ffmpeg is blowing up on the file conversion. Media is making it down to the local machine okay in the webm format.

I hope this is helpful for troubleshooting.

old-df38257622 commented 3 years ago

@Silverarmor - i'm not sure how to confirm this, nor get it? I'm using pip's spotDL. (Edit - figured out how to get dev from the Discord. Trying now) @aklajnert - The same playlist URL as shown in the original issue.

old-df38257622 commented 3 years ago

Same pytube error with Dev branch.

Silverarmor commented 3 years ago

Appears fixed in spotDL v3.3.1

  1. pip install pip-autoremove
  2. pip-autoremove spotdl
  3. pip cache purge
  4. pip install spotdl
old-df38257622 commented 3 years ago

Should I be opening an issue with pytube for the new error, then?

Silverarmor commented 3 years ago

@keli5 I can't reproduce your error in spotDL v3.3.1

fenixnet-net commented 3 years ago

@Silverarmor I'm still repro-ing the errors I listed above on the dev branch. I believe that's the original error being called as the issue.

old-df38257622 commented 3 years ago

I still get the pytube.exceptions.RegexMatchError: get_ytplayer_config: could not find match for config_patterns during download.

ghost commented 3 years ago
old-df38257622 commented 3 years ago

That gets us another new error, before downloading even begins, though I suspect it may be temporary.

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 130, in <module>
    console_entry_point()
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/__main__.py", line 110, in console_entry_point
    songObjList = get_playlist_tracks(request)
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/utils.py", line 82, in get_playlist_tracks
    song = SongObj.from_url(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/songObj.py", line 63, in from_url
    youtubeLink = SongObj.searchProvider(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 256, in search_and_get_best_match
    results = search_and_order_ytm_results(
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 150, in search_and_order_ytm_results
    results = __query_and_simplify(get_ytm_search_query(songName, songArtists))
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/search/provider.py", line 123, in __query_and_simplify
    searchResult = ytmApiClient.search(searchTerm, filter='videos')
  File "/home/letters/.local/lib/python3.8/site-packages/ytmusicapi/mixins/browsing.py", line 96, in search
    response = self._send_request(endpoint, body)
  File "/home/letters/.local/lib/python3.8/site-packages/ytmusicapi/ytmusic.py", line 116, in _send_request
    raise Exception(message + error)
Exception: Server returned HTTP 503: Service Unavailable.
The service is currently unavailable.
Exception ignored in: <function tqdm.__del__ at 0x7fb6759ed3a0>
Traceback (most recent call last):
  File "/home/letters/.local/lib/python3.8/site-packages/tqdm/std.py", line 1134, in __del__
  File "/home/letters/.local/lib/python3.8/site-packages/tqdm/std.py", line 1281, in close
  File "/home/letters/.local/lib/python3.8/site-packages/tqdm/std.py", line 1467, in display
  File "/home/letters/.local/lib/python3.8/site-packages/tqdm/std.py", line 1137, in __repr__
  File "/home/letters/.local/lib/python3.8/site-packages/spotdl/download/progressHandlers.py", line 23, in format_dict
  File "/home/letters/.local/lib/python3.8/site-packages/tqdm/std.py", line 1424, in format_dict
TypeError: cannot unpack non-iterable NoneType object

I'm converting it to a youtube playlist and downloading it with a playlist downloader.

Silverarmor commented 3 years ago

Closing issue as it is no longer reproducible in v3.3.1

  1. pip install pip-autoremove
  2. pip-autoremove spotdl
  3. pip cache purge
  4. pip install spotdl