get_playlist2 problem - Githubissues

embryo10 commented 7 years ago

While trying to get a playlist using the get_playlist method of pafy I get a couple of errors. pafy version 0.5.3.1 Using this YouTube link I get the following error:

  File "D:\Apps\DEV\PYTHON\Python27\lib\site-packages\pafy\playlist.py", line 193, in __iter__
    views=vextra['statistics'].get('viewCount',0),
KeyError: 'statistics'

It seems there is a missing key there. I change the for loop in the playlist.py like this:

            for v, vextra in zip(playlistitems['items'], wdata['items']):
                vid_data = dict(
                    title=v['snippet']['title'],
                    author=v['snippet']['channelTitle'],
                    thumbnail=v['snippet'].get('thumbnails', {}
                        ).get('default', {}).get('url'),
                    description=v['snippet']['description'],
                    length_seconds=parseISO8591(
                        vextra['contentDetails']['duration']),
                    category=get_categoryname(vextra['snippet']['categoryId']),
                # 2fix: --------------------------------------------------------
                #     views=vextra['statistics'].get('viewCount',0),
                #     likes=vextra['statistics'].get('likeCount',0),
                #     dislikes=vextra['statistics'].get('dislikeCount',0),
                #     comments=vextra['statistics'].get('commentCount',0),
                )
                try:
                    vid_data['views'] = vextra['statistics'].get('viewCount',0),
                    vid_data['likes'] = vextra['statistics'].get('likeCount',0),
                    vid_data['dislikes'] = vextra['statistics'].get('dislikeCount',0),
                    vid_data['comments'] = vextra['statistics'].get('commentCount',0)
                except KeyError as err:
                    if err.args[0] == 'statistics':
                        vextra['statistics'] = dict()
                    else:
                        print 'Error in pafy/playlist:', err
                # 2fix: --------------------------------------------------------
                try:
                    pafy_obj = new(v['snippet']['resourceId']['videoId'],
                            basic=self._basic, gdata=self._gdata,
                            size=self._size, callback=self._callback)

                except IOError as e:
                    self.callback("%s: %s" % (v['title'], e.message))
                    continue

                pafy_obj.populate_from_playlist(vid_data)
                items.append(pafy_obj)
                self._callback("Added video: %s" % vid_data['title'])
                yield pafy_obj

Solved this, but then I get another error:

  File "D:\Apps\DEV\PYTHON\Python27\lib\site-packages\pafy\playlist.py", line 219, in __iter__
    pafy_obj.populate_from_playlist(vid_data)
  File "D:\Apps\DEV\PYTHON\Python27\lib\site-packages\pafy\backend_shared.py", line 414, in populate_from_playlist
    self._viewcount = "".join(re.findall(r"\d", pl_data.get("views", "0")))
  File "D:\Apps\DEV\PYTHON\Python27\lib\re.py", line 181, in findall
    return _compile(pattern, flags).findall(string)
TypeError: expected string or buffer

I found out that "views" is getting me a tuple instead of a string. A fast hack was to change the populate_from_playlist method like this:

    def populate_from_playlist(self, pl_data):
        """ Populate Pafy object with items fetched from playlist data. """
        self._title = pl_data.get("title")
        self._author = pl_data.get("author")
        self._length = int(pl_data.get("length_seconds", 0))
        self._rating = pl_data.get("rating", 0.0)
        # 2fix: --------------------------------------------------------
        if type(pl_data.get("views", "0")) == tuple:
            pl_data['views'] = pl_data['views'][0]
        # 2fix: --------------------------------------------------------
        self._viewcount = "".join(re.findall(r"\d", pl_data.get("views", "0")))
        self._viewcount = int(self._viewcount)
        self._description = pl_data.get("description")

This did the trick and let me get the playlist OK.

ids1024 commented 7 years ago

Odd. It seems statistics is present for most of the videos, but it reaches one where it is missing:

{'contentDetails': {'caption': 'false',
                    'definition': 'sd',
                    'dimension': '2d',
                    'duration': 'PT4M23S',
                    'licensedContent': True,
                    'projection': 'rectangular'},
 'etag': '"m2yskBQFythfE4irbTIeOgYYfBU/oTuBkTF3zirKQ85Cj0EgM78FUF0"',
 'id': 'iQ4D273C7Ac',
 'kind': 'youtube#video',
 'snippet': {'categoryId': '10',
             'channelId': 'UCxnQCDKv8K2XYuqp9DgqRaA',
             'channelTitle': 'TruThoughtsRecords',
             'description': 'Tru Thoughts Covers is a mighty selection of the '
                            'most pivotal and interesting cover versions to '
                            'have peppered Tru Thoughts ten year history of '
                            'releases, boasting many crossover hits.',
             'liveBroadcastContent': 'none',
             'localized': {'description': 'Tru Thoughts Covers is a mighty '
                                          'selection of the most pivotal and '
                                          'interesting cover versions to have '
                                          'peppered Tru Thoughts ten year '
                                          'history of releases, boasting many '
                                          'crossover hits.',
                           'title': 'Nostalgia 77 Feat Alice Russell - Seven '
                                    'Nation Army (The White Stripes Cover)'},
             'publishedAt': '2009-08-04T10:49:57.000Z',
             'tags': ['Nostalgia 77',
                      'Alice Russell',
                      'Seven Nation Army',
                      'The White Stripes',
                      'Tru Thoughts'],
             'thumbnails': {'default': {'height': 90,
                                        'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/default.jpg',
                                        'width': 120},
                            'high': {'height': 360,
                                     'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/hqdefault.jpg',
                                     'width': 480},
                            'medium': {'height': 180,
                                       'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/mqdefault.jpg',
                                       'width': 320}},
             'title': 'Nostalgia 77 Feat Alice Russell - Seven Nation Army '
                      '(The White Stripes Cover)'}}

embryo10 commented 7 years ago

Most of them are OK but I think that there are 5 of the problematic ones in the example list:

Nostalgia 77 Feat Alice Russell - Seven Nation Army (The White Stripes Cover)
Jazz and 80s - Boys Dont Cry
Jazz Chill - Tainted Love
Eliza Doolittle | Creep - Radiohead Acoustic Cover | Eliza Covers
Nouvelle Vague - Ever Fallen In Love

On 9/4/2017 1:19 πμ, Ian Douglas Scott wrote:

Odd. It seems |statistics| is present for most of the videos, but it reaches one where it is missing:

|{'contentDetails': {'caption': 'false', 'definition': 'sd', 'dimension': '2d', 'duration': 'PT4M23S', 'licensedContent': True, 'projection': 'rectangular'}, 'etag': '"m2yskBQFythfE4irbTIeOgYYfBU/oTuBkTF3zirKQ85Cj0EgM78FUF0"', 'id': 'iQ4D273C7Ac', 'kind': 'youtube#video', 'snippet': {'categoryId': '10', 'channelId': 'UCxnQCDKv8K2XYuqp9DgqRaA', 'channelTitle': 'TruThoughtsRecords', 'description': 'Tru Thoughts Covers is a mighty selection of the ' 'most pivotal and interesting cover versions to ' 'have peppered Tru Thoughts ten year history of ' 'releases, boasting many crossover hits.', 'liveBroadcastContent': 'none', 'localized': {'description': 'Tru Thoughts Covers is a mighty ' 'selection of the most pivotal and ' 'interesting cover versions to have ' 'peppered Tru Thoughts ten year ' 'history of releases, boasting many ' 'crossover hits.', 'title': 'Nostalgia 77 Feat Alice Russell - Seven ' 'Nation Army (The White Stripes Cover)'}, 'publishedAt': '2009-08-04T10:49:57.000Z', 'tags': ['Nostalgia 77', 'Alice Russell', 'Seven Nation Army', 'The White Stripes', 'Tru Thoughts'], 'thumbnails': {'default': {'height': 90, 'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/default.jpg', 'width': 120}, 'high': {'height': 360, 'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/hqdefault.jpg', 'width': 480}, 'medium': {'height': 180, 'url': 'https://i.ytimg.com/vi/iQ4D273C7Ac/mqdefault.jpg', 'width': 320}}, 'title': 'Nostalgia 77 Feat Alice Russell - Seven Nation Army ' '(The White Stripes Cover)'}} |

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mps-youtube/pafy/issues/169#issuecomment-292749571, or mute the thread https://github.com/notifications/unsubscribe-auth/AISEw0K2sYV6p4KOFTbH0gS-HMBUwgm4ks5ruAfkgaJpZM4M31eW.

ids1024 commented 7 years ago

PR #171 should fix this.

mps-youtube / pafy

get_playlist2 problem #169