mps-youtube / pafy

Python library to download YouTube content and retrieve metadata
1.38k stars 315 forks source link

KeyError: dislike_count #306

Open dkao1978 opened 2 years ago

dkao1978 commented 2 years ago

When starting up mps-youtube, pafy is throwing an exception for unknown key for 'dislike_count'. Youtube is now hiding the thumbs down count, so I'm guessing the API data is now gone.

Manually setting the variable to 0 makes the problem go away. It seems like the metadata should be removed since its not accessible anymore.

Traceback (most recent call last):
  File "/Users/foo/.pyenv/versions/3.7.4/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/Users/foo/.pyenv/versions/3.7.4/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/mps_youtube/streams.py", line 180, in _preload
    streamlist = get(song)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/mps_youtube/streams.py", line 51, in get
    p = util.get_pafy(vid, force=force, callback=callback)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/mps_youtube/util.py", line 221, in get_pafy
    p = pafy.new(ytid, callback=callback_fn)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/pafy/pafy.py", line 124, in new
    return Pafy(url, basic, gdata, size, callback, ydl_opts=ydl_opts)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/pafy/backend_youtube_dl.py", line 31, in __init__
    super(YtdlPafy, self).__init__(*args, **kwargs)
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/pafy/backend_shared.py", line 97, in __init__
    self._fetch_basic()
  File "/Users/foo/.local/pipx/venvs/mps-youtube/lib/python3.7/site-packages/pafy/backend_youtube_dl.py", line 54, in _fetch_basic
    self._dislikes = self._ydl_info['dislike_count']
KeyError: 'dislike_count'
dkao1978 commented 2 years ago

Looks like there's a PR for it:

https://github.com/mps-youtube/pafy/pull/305

pb581 commented 2 years ago

I noticed it's kind of hit-or-miss: the like/dislike count metadata is there, but sometimes it gets fetched, sometimes it doesn't; could be something to do with privacy settings and/or is the case for running livestreams where the metadata isn't fetched properly sometimes.

The issue was fixed with the merge of pull #288 and it's precisely the fix you mention: initialize 0 if said metadata isn't fetched properly. Unfortunately, the last pull committed in latest wheel release (0.5.5) is #235, so the bugfix is to be done manually till next release (if there will be one!)

I agree however that the like/dislike (also a few other useless metadata) ought to be removed altogether.

rubyFeedback commented 2 years ago

I just had the very same problem. I also installed an extension in firefox to still see the dislike_count because I want to see how many people dislike a video; very malicious of Google to remove it.

But I also need to find out the duration of video length, without the Google API (I am not going to use API keys and depend on google in any of my scripts). After searching I found pafy; unfortunately I was just stopped with the same error:

Traceback (most recent call last):
  File "/Depot/jj/yo.py", line 4, in <module>
    video = pafy.new(url)
  File "/usr/lib64/python3.9/site-packages/pafy/pafy.py", line 124, in new
    return Pafy(url, basic, gdata, size, callback, ydl_opts=ydl_opts)
  File "/usr/lib64/python3.9/site-packages/pafy/backend_youtube_dl.py", line 31, in __init__
    super(YtdlPafy, self).__init__(*args, **kwargs)
  File "/usr/lib64/python3.9/site-packages/pafy/backend_shared.py", line 97, in __init__
    self._fetch_basic()
  File "/usr/lib64/python3.9/site-packages/pafy/backend_youtube_dl.py", line 54, in _fetch_basic
    self._dislikes = self._ydl_info['dislike_count']
KeyError: 'dislike_count'

Would be nice if pafy could be adapted to tolerate users using dislike_count (or I suppose Google changed something internally, so pafy may have to be adjusted as well anyway)..

SandeepKanekal commented 2 years ago

I had gotten the same error many times, since YouTube has removed dislikes, pafy cannot fetch it for us. You will have to comment line 54 in pafy.backend_youtube_dl. Or if you are hosting, you can download the zip of the repository and then comment the line. Check out this code, you may get an idea of what to do, if you desperately need the dislike count.

Kirandevraj commented 2 years ago

I faced the same issue. I installed the package directly from git using pip install git+https://github.com/mps-youtube/pafy.git. This has fixed the issue.

omegaab2 commented 2 years ago

https://github.com/mps-youtube/pafy.git

I try this way on heroku it's gives me this error:

Collecting pafy@ git+https://github.com/mps-youtube/pafy.git@0580c7247e74f4caa97154ad4fd6763e991198f8
         Cloning https://github.com/mps-youtube/pafy.git (to revision 0580c7247e74f4caa97154ad4fd6763e991198f8) to /tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a
         Running command git clone --filter=blob:none --quiet https://github.com/mps-youtube/pafy.git /tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a
         Running command git rev-parse -q --verify 'sha^0580c7247e74f4caa97154ad4fd6763e991198f8'
         Running command git fetch -q https://github.com/mps-youtube/pafy.git 0580c7247e74f4caa97154ad4fd6763e991198f8
         Resolved https://github.com/mps-youtube/pafy.git to commit 0580c7247e74f4caa97154ad4fd6763e991198f8
         Preparing metadata (setup.py): started
         Preparing metadata (setup.py): finished with status 'error'
         error: subprocess-exited-with-error

         × python setup.py egg_info did not run successfully.
         │ exit code: 1
         ╰─> [17 lines of output]
             Traceback (most recent call last):
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/pafy.py", line 48, in <module>
                 import youtube_dl
             ModuleNotFoundError: No module named 'youtube_dl'

             During handling of the above exception, another exception occurred:

             Traceback (most recent call last):
               File "<string>", line 2, in <module>
               File "<pip-setuptools-caller>", line 34, in <module>
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/setup.py", line 13, in <module>
                 from pafy import __version__
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/__init__.py", line 7, in <module>
                 from .pafy import new
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/pafy.py", line 51, in <module>
                 raise ImportError(
             ImportError: pafy: youtube-dl not found; you can use the internal backend by setting the environmental variable PAFY_BACKEND to "internal". It is not enabled by default because it is not as well maintained as the youtube-dl backend.
             [end of output]

         note: This error originates from a subprocess, and is likely not a problem with pip.
       error: metadata-generation-failed

       × Encountered error while generating package metadata.
       ╰─> See above for output.

       note: This is an issue with the package mentioned above, not pip.
       hint: See above for details.
 !     Push rejected, failed to compile Python app.
 !     Push failed
SandeepKanekal commented 2 years ago

https://github.com/mps-youtube/pafy.git

I try this way on heroku it's gives me this error:

Collecting pafy@ git+https://github.com/mps-youtube/pafy.git@0580c7247e74f4caa97154ad4fd6763e991198f8
         Cloning https://github.com/mps-youtube/pafy.git (to revision 0580c7247e74f4caa97154ad4fd6763e991198f8) to /tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a
         Running command git clone --filter=blob:none --quiet https://github.com/mps-youtube/pafy.git /tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a
         Running command git rev-parse -q --verify 'sha^0580c7247e74f4caa97154ad4fd6763e991198f8'
         Running command git fetch -q https://github.com/mps-youtube/pafy.git 0580c7247e74f4caa97154ad4fd6763e991198f8
         Resolved https://github.com/mps-youtube/pafy.git to commit 0580c7247e74f4caa97154ad4fd6763e991198f8
         Preparing metadata (setup.py): started
         Preparing metadata (setup.py): finished with status 'error'
         error: subprocess-exited-with-error

         × python setup.py egg_info did not run successfully.
         │ exit code: 1
         ╰─> [17 lines of output]
             Traceback (most recent call last):
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/pafy.py", line 48, in <module>
                 import youtube_dl
             ModuleNotFoundError: No module named 'youtube_dl'

             During handling of the above exception, another exception occurred:

             Traceback (most recent call last):
               File "<string>", line 2, in <module>
               File "<pip-setuptools-caller>", line 34, in <module>
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/setup.py", line 13, in <module>
                 from pafy import __version__
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/__init__.py", line 7, in <module>
                 from .pafy import new
               File "/tmp/pip-install-gkbgos_k/pafy_673c51aab0f74a8884fb7f4e8c42414a/pafy/pafy.py", line 51, in <module>
                 raise ImportError(
             ImportError: pafy: youtube-dl not found; you can use the internal backend by setting the environmental variable PAFY_BACKEND to "internal". It is not enabled by default because it is not as well maintained as the youtube-dl backend.
             [end of output]

         note: This error originates from a subprocess, and is likely not a problem with pip.
       error: metadata-generation-failed

       × Encountered error while generating package metadata.
       ╰─> See above for output.

       note: This is an issue with the package mentioned above, not pip.
       hint: See above for details.
 !     Push rejected, failed to compile Python app.
 !     Push failed

@omegaab2, you have to include youtube-dl in your requirements.txt file.

omegaab2 commented 2 years ago

git+https://github.com/mps-youtube/pafy.git

i fix it by install library from my code because pafy needs youtube-dl installed before but heroku install all of them together so you can install pafy from your code


import subprocess, sys

def install(package):
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])

try:
    import pafy

except Exception as error:
    print("Error: ", error)
    install("git+https://github.com/mps-youtube/pafy.git")
    import pafy