newgrounds.com - '403: Forbidden' error on creator / profile / account pages

ghost commented 1 year ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.03.04 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region

No response

Provide a description that is worded well enough to be understood

example URL: https://doki.newgrounds.com/movies

attempting to download prints the error:

ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>)

The main creator pages (e.g. https://doki.newgrounds.com) are reported as 'unsupported':

[generic] Extracting URL: https://doki.newgrounds.com/
[generic] doki.newgrounds: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] doki.newgrounds: Extracting information
ERROR: Unsupported URL: https://doki.newgrounds.com/

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: [-vU, --paths, /root/yt-dlp/, https://doki.newgrounds.com/movies]
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d]
[debug] Lazy loading extractors is disabled
[debug] Python 3.11.3 (CPython x86_64 64bit) - Linux-6.3.6-200.fc38.x86_64-x86_64-with-glibc2.37 (OpenSSL 3.0.9 30 May 2023, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (fdk,setts), ffprobe 6.0
[debug] Optional libraries: Cryptodome-3.17.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.03.04, Current version: stable@2023.03.04
yt-dlp is up to date (stable@2023.03.04)
[Newgrounds:user] Extracting URL: https://doki.newgrounds.com/movies
[download] Downloading playlist: doki
[Newgrounds:user] doki: Downloading page 1
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: Forbidden>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 821, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3742, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

[Newgrounds:user] Playlist doki: Downloading 1 items of 0
[download] Finished downloading playlist: doki

bashonly commented 1 year ago

note for devs: playlist extractor tries to request /movies/page/1 which doesn't work in browser either

october262 commented 1 year ago

go to doki's home page - latest movies - right click & copy the subscribe to RSS feed URL , you can download 10 videos with that link.

yt-dlp / yt-dlp