Open jellofan opened 7 years ago
Mr-Jake
commented
7 years ago This was fixed by @outcoldman: https://github.com/rg3/youtube-dl/pull/11258 I compiled youtube-dl with the fix and Comcast_SSO works with aetv.
Wait till it gets merged, or you could always compile it yourself.
jellofan
commented
7 years ago I pulled the repo, applied the patch, compiled, and still have the issue. I think this is something different. Using --write-pages I see the page returned after login is a "Recovery Options" page.
Redacted version is here (removed e-mail and Name) recovery_options_redacted.txt
Mr-Jake
commented
7 years ago This is what I get with the applied patch and your example link. I compiled youtube-dl 2016-12-22.
C:\youtube-dl\youtube-dl.exe "http://www.aetv.com/shows/going-si-ral/season-1/episode-6" --ap-mso Comcast_SSO --ap-username <omitted> --ap-password <omitted> -F
[aenetworks] going-si-ral/season-1/episode-6: Downloading webpage
[aenetworks] 838677059721: Downloading JSON metadata
[aenetworks] 838677059721: Downloading Provider Redirect Page
[aenetworks] 838677059721: Confirming auto login
[aenetworks] 838677059721: Retrieving Session
[aenetworks] 838677059721: Retrieving Authorization Token
[aenetworks] 838677059721: Retrieving Media Token
[aenetworks] 838677059721: Downloading SMIL data
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[aenetworks] 838677059721: Checking video URL
[info] Available formats for 838677059721:
format code extension resolution note
http-162 mp4 320x180 162k
http-326 mp4 320x180 326k
http-526 mp4 448x252 526k
http-826 mp4 512x288 826k
http-1126 mp4 640x360 1126k
http-1426 mp4 768x432 1426k
http-1926 mp4 960x540 1926k
http-2618 mp4 1280x720 2618k (best)
I downloaded the youtube-dl source code. Found and opened the file adobepass.py. Copy and pasted in the added lines of code by outcoldman. Then compiled and that solved my Comcast issues. I'm using Windows not Linux.
Perhaps it is a different issue. Just wanted to point you to the fix in case it would help.
jellofan
commented
7 years ago I think my issue is different since I see the expected:
[aenetworks] 838677059721: Downloading Provider Redirect Page
[aenetworks] 838677059721: Logging in
[aenetworks] 838677059721: Retrieving Session
And I see my name & personal (non-comcast) e-mail in the login.comcast.net_login.dump. So I'm fairly confident I've been authenticated.
From an internet search it seems that in some cases Comcast presents a Recovery Options page immediately after logging in (see: http://forums.xfinity.com/t5/Email-Web-Browsing/Recvery-Options-Page/td-p/2687138). What is strange is that I don't see this page when using a browser, just when using youtube-dl.
The recovery options page (which I previously attached) contains a form, and one of the buttons is "askMeLater", so I think the solution to this issue is in adobepass.py after the login form is posted to add logic to check for the recovery page, i.e. just before the check for the "Resume" button, and submit the "askMeLater" form.
But what I'm unsure of is how to populate the "data-tracking" portion of the button:
<button
class="secondary button cancel"
type="submit"
id="askMeLater"
name="_eventId"
value="askMeLater"
data-tracking
>Ask me later</button>
ForTehLose
commented
7 years ago @jellofan so I know this isn't a permanent fix, but your issue is with the recovery options page. I had the same issue, if you have previously signed in and clicked remind me later you can't review the page on that user agent anymore. Comcast simply wont prompt you. A short term fix is to try signing in with a different browser, and thus a different agent, and verify your recovery options.
It will no longer prompt you on any user agents including the one from youtubedl
joetex72
commented
7 years ago 
GordLacey
commented
6 years ago I'm having the same problem (with 2018.04.03), but on Cartoon Network. Could it be that there's a CAPTCHA on the Comcast login page that's not being filled out?
youtube-dl --ap-mso Comcast_SSO --ap-username PRIVATE@comcast.net --ap-password PRIVATE https://www.cartoonnetwork.com/video/nexo-knights/weekend-at-halberts-episode.html --verbose [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--ap-mso', u'Comcast_SSO', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'https://www.cartoonnetwork.com/video/nexo-knights/weekend-at-halberts-episode.html', u'--verbose'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.04.03 [debug] Python version 2.7.10 (CPython) - Darwin-16.7.0-x86_64-i386-64bit [debug] exe versions: avconv 12.1, avprobe 12.1, ffmpeg 3.3.3, ffprobe 3.3.3 [debug] Proxy map: {} [CartoonNetwork] weekend-at-halberts: Downloading webpage [CartoonNetwork] 598df84d240214953c1bb13eaeef2e369a59fb46: Downloading XML [CartoonNetwork] 598df84d240214953c1bb13eaeef2e369a59fb46: Downloading Provider Redirect Page [CartoonNetwork] 598df84d240214953c1bb13eaeef2e369a59fb46: Logging in [CartoonNetwork] 598df84d240214953c1bb13eaeef2e369a59fb46: Retrieving Session ERROR: Unable to download webpage: HTTP Error 401: Unauthorized (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 519, in _request_webpage return self._downloader.urlopen(url_or_request) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2199, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 475, in error return self._call_chain(args) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(args) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 558, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
tv21
commented
5 years ago This was fixed by @outcoldman: #11258 I compiled youtube-dl with the fix and Comcast_SSO works with aetv.
Wait till it gets merged, or you could always compile it yourself.
Actually that fix stopped working due to a minor URL change. For those running from source code or compiling your own, the quick fix for Comcast Xfinity is to use this version of adobepass.py:
https://raw.githubusercontent.com/bhamboss/youtube-dl/patch-1/youtube_dl/extractor/adobepass.py
That is from https://github.com/bhamboss/youtube-dl/blob/patch-1/youtube_dl/extractor/adobepass.py and I can confirm this works with Xfinity's new login if you are coming in from an Xfinity IP address.
Just to be clear, Xfinity authenticates differently if you are coming in from one of their IP addresses vs. if you are coming in from the IP of some other provider. I tested the above patch coming from an Xfinity IP. I did NOT test it from another provider's IP.
If I run diff -u on the original and patched files I get this:
$ diff -u adobepass.py.original adobepass.py.patched
--- adobepass.py.original 2018-10-17 01:22:56.000000000 -0400
+++ adobepass.py.patched 2019-04-23 08:53:36.000000000 -0400
@@ -1433,6 +1433,14 @@
provider_redirect_page, 'oauth redirect')
self._download_webpage(
oauth_redirect_url, video_id, 'Confirming auto login')
+ elif 'automatically signed in with' in provider_redirect_page:
+ # Seems like comcast is rolling up new way of automatically signing customers
+ oauth_redirect_url = self._html_search_regex(
+ r'continue:\s*"(https://oauth.xfinity.com/oauth/authorize\?.+)"',
+ provider_redirect_page, 'oauth redirect (signed)')
+ # Just need to process the request. No useful data comes back
+ self._download_webpage(
+ oauth_redirect_url, video_id, 'Confirming auto login')
else:
if '<form name="signin"' in provider_redirect_page:
provider_login_page_res = provider_redirect_page_res
sufr3ak
commented
4 years ago Have you noticed the url change again with Comcast?
RianovRV1
commented
3 years ago Looks like comcast is not working again as of now
Please follow the guide below
xinto all the boxes [ ] relevant to your issue (like that [x])Make sure you are using the latest version: run
youtube-dl --versionand ensure your version is 2016.12.22. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.Before submitting an issue make sure you have:
What is the purpose of your issue?
The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue
If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add
-vflag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):Description of your issue, suggested solution and other information
Unable to download aetv.com videos using Comcast SSO. I tried several videos (including the one with --verbose info attached)