ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.17k stars 9.93k forks source link

pluralsight download video Failed to parse JSON #30818

Open nisamo opened 2 years ago

nisamo commented 2 years ago

youtube-dl --cookies cookies.txt --verbose --sleep-interval 600 https://app.pluralsight.com/library/courses/django-angularjs-web-development/table-of-contents

i get error:

C:\youtube-dl>youtube-dl --cookies cookies.txt --verbose --sleep-interval 600 https://app.pluralsight.com/library/courses/django-angularjs-web-development/table-of-contents
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', '--sleep-interval', '600', 'https://app.pluralsight.com/library/courses/django-angularjs-web-development/table-of-contents']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.14393
[debug] exe versions: none
[debug] Proxy map: {}
[pluralsight:course] django-angularjs-web-development: Downloading JSON metadata
[pluralsight:course] django-angularjs-web-development: Downloading JSON metadata
ERROR: django-angularjs-web-development: Failed to parse JSON  (caused by ValueError('Expecting value: line 1 column 1 (char 0)',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 906, in _parse_json
  File "C:\Python\Python34\lib\json\__init__.py", line 318, in loads
  File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode
  File "C:\Python\Python34\lib\json\decoder.py", line 361, in raw_decode
ValueError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 906, in _parse_json
  File "C:\Python\Python34\lib\json\__init__.py", line 318, in loads
  File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode
  File "C:\Python\Python34\lib\json\decoder.py", line 361, in raw_decode
ValueError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 815, in wrapper
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 836, in __extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 534, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\pluralsight.py", line 467, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\pluralsight.py", line 93, in _download_course
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 899, in _download_json
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 883, in _download_json_handle
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 910, in _parse_json
youtube_dl.utils.ExtractorError: django-angularjs-web-development: Failed to parse JSON  (caused by ValueError('Expecting value: line 1 column 1 (char 0)',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
dirkf commented 2 years ago

Duplicate of #30765, except Py3 instead of Py2.

If you had reported this using the Broken site support template, you would have been prompted to check existing issues.

nisamo commented 2 years ago

no solve this problem?

dirkf commented 2 years ago

The original report was made a few days ago, but no Pluralsight user has been able to work on debugging the problem yet.

Generally, commercial courseware sites are difficult to support because they change the content structure in ways that only account holders can identify and are prone to deploy specific measures against tools like yt-dl.

valioiv commented 2 years ago

Hi, any resolution on the issue?

thisaccountname commented 2 years ago

Also running into this issue. I've clicked through a long chain of responses mentioning that issues are duplicates but no proposed fix. Any ideas?

dirkf commented 2 years ago

Comment https://github.com/ytdl-org/youtube-dl/issues/30818#issuecomment-1087836864 still applies. If any progress is being made issue #30765 will be updated. See also https://github.com/yt-dlp/yt-dlp/issues/4570.

thisaccountname commented 2 years ago

@dirkf I am attempting to look into this but can't seem to write the JSON it's failing to parse?

I've tried --write-pages --dump-pages --dump-json - am I barking up the wrong tree?

dirkf commented 2 years ago

--print-traffic will show the protocol exchanges between yt-dl and the webserver. --write-pages will write a copy of each page fetched from the webserver.

You can cross-check these with what you see in your browser's development tools.

In OP's log what's happening is that the primary API is failing and the fallback raises the exception (youtube_dl/extractor/pluralsight.py:85 ff.). This would give better diagnostics:

     def _download_course(self, course_id, url, display_id):
         try:
             return self._download_course_rpc(course_id, url, display_id)
-        except ExtractorError:
+        except ExtractorError as e:
             # Old API fallback
-            return self._download_json(
-                'https://app.pluralsight.com/player/user/api/v1/player/payload',
-                display_id, data=urlencode_postdata({'courseId': course_id}),
-                headers={'Referer': url})
+            try:
+                return self._download_json(
+                    'https://app.pluralsight.com/player/user/api/v1/player/payload',
+                    display_id, data=urlencode_postdata({'courseId': course_id}),
+                    headers={'Referer': url})
+            except ExtractorError as still_e:
+                self.report_warning('Fallback course API failed: ' + error_to_compat_str(still_e))
+                raise e

For detailed investigation, check out the code and stick

        import pdb; pdb.set_trace()

ahead of the first try: to get it to break there. Run the code using python -m youtube_dl ..., from the directory that contains the checked-out code (eg, contains the youtube_dl directory), instead of youtube-dl ....

JamiePattison commented 2 years ago

@dirkf I am attempting to look into this but can't seem to write the JSON it's failing to parse?

I've tried --write-pages --dump-pages --dump-json - am I barking up the wrong tree?

Did you get any further with this?

dually8 commented 2 years ago

It's because the backup no longer returns JSON, just a simple "OK". The rpc one gives a 403 forbidden no matter what. image