jwplayer/wimp: TypeError: argument of type 'NoneType' is not iterable

The-Compiler commented 5 years ago

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2019.08.13
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl -v 'https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/'
[debug] System config: []
[debug] User config: ['--mark-watched', '--output', '%(uploader)s - %(title)s %(id)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.13
[debug] Python version 3.7.4 (CPython) - Linux-5.2.8-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.2, ffprobe 4.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[Wimp] rossi-vs-lorenzo-incredible-final-lap: Downloading webpage
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2019.8.13', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3.7/site-packages/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/lib/python3.7/site-packages/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 2010, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/wimp.py", line 46, in _real_extract
    webpage, video_id, require_title=False)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 2645, in _extract_jwplayer_data
    jwplayer_data, video_id, *args, **kwargs)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 2651, in _parse_jwplayer_data
    if 'playlist' not in jwplayer_data:
TypeError: argument of type 'NoneType' is not iterable

Description

URL: https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/

bitraid commented 5 years ago

Hi @The-Compiler , it seems that the generic extractor (--force-generic-extractor) now works fine.

The-Compiler commented 5 years ago

No difference for me:

[debug] System config: []
[debug] User config: ['--mark-watched', '--output', '%(uploader)s - %(title)s %(id)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['-v', '--force-generic-extractor', 'http://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.13
[debug] Python version 3.7.4 (CPython) - Linux-5.2.8-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.2, ffprobe 4.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] rossi-vs-lorenzo-incredible-final-lap: Requesting header
[redirect] Following redirect to https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/
[Wimp] rossi-vs-lorenzo-incredible-final-lap: Downloading webpage
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 11, in <module>
    load_entry_point('youtube-dl==2019.8.13', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3.7/site-packages/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/lib/python3.7/site-packages/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 2010, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 807, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 868, in process_ie_result
    extra_info=extra_info)
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/wimp.py", line 46, in _real_extract
    webpage, video_id, require_title=False)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 2645, in _extract_jwplayer_data
    jwplayer_data, video_id, *args, **kwargs)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 2651, in _parse_jwplayer_data
    if 'playlist' not in jwplayer_data:
TypeError: argument of type 'NoneType' is not iterable

bitraid commented 5 years ago

For some reason, it still uses wimp for you:

[redirect] Following redirect to https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/
[Wimp] rossi-vs-lorenzo-incredible-final-lap: Downloading webpage

This is my log (same parameters):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.wimp.com/rossi-vs-lorenzo-incredible-final-lap/', '--force-generic-extractor', '-v', '--mark-watched', '--output', '%(uploader)s - %(title)s %(id)s.%(ext)s']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.13
[debug] Git HEAD: def849e0e
[debug] Python version 3.7.4 (CPython) - Linux-5.2.7-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.2, ffprobe 4.2, phantomjs ., rtmpdump 2.4
[debug] Proxy map: {}
[generic] rossi-vs-lorenzo-incredible-final-lap: Requesting header
WARNING: Forcing on generic information extractor.
[generic] rossi-vs-lorenzo-incredible-final-lap: Downloading webpage
[generic] rossi-vs-lorenzo-incredible-final-lap: Extracting information
[download] Downloading playlist: Rossi vs. Lorenzo: Incredible final lap.
[generic] playlist Rossi vs. Lorenzo: Incredible final lap.: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://cdn2.wimp.com/videos/3d16ad016140a71380bc612e99aac571_lap.flv.hq.mp4?_=1'
[download] Destination: NA - Rossi vs. Lorenzo - Incredible final lap. rossi-vs-lorenzo-incredible-final-lap.mp4

[download]   0.0% of 32.99MiB at Unknown speed ETA Unknown ETA
[download]   0.0% of 32.99MiB at Unknown speed ETA Unknown ETA
[download]   0.0% of 32.99MiB at Unknown speed ETA Unknown ETA
[download]   0.0% of 32.99MiB at Unknown speed ETA Unknown ETA
[download]   0.1% of 32.99MiB at 342.77KiB/s ETA 01:38
[download]   0.2% of 32.99MiB at 365.39KiB/s ETA 01:32
[download]   0.4% of 32.99MiB at 498.48KiB/s ETA 01:07
[download]   0.8% of 32.99MiB at 767.06KiB/s ETA 00:43
[download]   1.5% of 32.99MiB at  1.27MiB/s ETA 00:25
[download]   3.0% of 32.99MiB at  1.42MiB/s ETA 00:22
[download]   6.1% of 32.99MiB at  2.29MiB/s ETA 00:13
[download]  12.1% of 32.99MiB at  3.18MiB/s ETA 00:09
[download]  24.2% of 32.99MiB at  4.16MiB/s ETA 00:06
[download]  36.4% of 32.99MiB at  4.44MiB/s ETA 00:04
[download]  48.5% of 32.99MiB at  4.56MiB/s ETA 00:03
[download]  60.6% of 32.99MiB at  4.56MiB/s ETA 00:02
[download]  72.7% of 32.99MiB at  4.72MiB/s ETA 00:01
[download]  84.9% of 32.99MiB at  4.76MiB/s ETA 00:01
[download]  97.0% of 32.99MiB at  4.79MiB/s ETA 00:00
[download] 100.0% of 32.99MiB at  4.80MiB/s ETA 00:00
[download] 100% of 32.99MiB in 00:07
[download] Finished downloading playlist: Rossi vs. Lorenzo: Incredible final lap.

Can you try #22091?

bitraid commented 5 years ago

The reason for the redirect (and wimp usage) is the use of http instead of https for the url.

The-Compiler commented 5 years ago

Ah, right! I tried that because I wanted to find out if the page's reply is actually the video (and not some error page or something).

Works fine with --force-generic-extractor and https indeed - but isn't that kind of a bug on its own?

Hrxn commented 5 years ago

@bitraid What about videos on Wimp embedded from YouTube or Instagram? Does this work with --force-generic-extractor as well, did you test this?

bitraid commented 5 years ago

@The-Compiler The generic extractor should use the detected extractor on redirects, but when used with --force-generic-exctractor, i think you are right, it should stick with it. @dstftw , @remitamine What do you think?

@Hrxn Yes, they work.

ytdl-org / youtube-dl

jwplayer/wimp: TypeError: argument of type 'NoneType' is not iterable #22088

Verbose log

Description