search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.34k stars 10.03k forks source link

crackle.com movies are not supported #11775

Open marafa opened 7 years ago

marafa commented 7 years ago

Please follow the guide below

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.01.10. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

youtube-dl -v http://www.crackle.com/7-seconds
[debug] System config: ['--prefer-free-formats']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'http://www.crackle.com/7-seconds']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.10
[debug] Python version 3.5.2 - Linux-4.9.3-200.fc25.x86_64-x86_64-with-fedora-25-Twenty_Five
[debug] exe versions: ffmpeg 3.1.6, ffprobe 3.1.6
[debug] Proxy map: {}
[generic] 7-seconds: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 7-seconds: Downloading webpage
[generic] 7-seconds: Extracting information
ERROR: Unsupported URL: http://www.crackle.com/7-seconds
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 693, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 359, in extract
    return self._real_extract(url)
  File "/usr/lib/python3.5/site-packages/youtube_dl/extractor/generic.py", line 2507, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: http://www.crackle.com/7-seconds

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Description of your issue, suggested solution and other information

Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible. If work on your issue requires account credentials please provide them or explain how one can obtain them.

youtube-dl already supports the TV section of crackle.com. However, it does not support the movies section

yan12125 commented 7 years ago

I got blank screens from http://www.crackle.com/7-seconds in both Firefox and Chrome. Is it geo-restricted?

marafa commented 7 years ago

I believe so . Yes

On Jan 19, 2017 11:27, "Yen Chi Hsuan" notifications@github.com wrote:

I got blank screens from http://www.crackle.com/7-seconds in both Firefox and Chrome. Is it geo-restricted?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rg3/youtube-dl/issues/11775#issuecomment-273823663, or mute the thread https://github.com/notifications/unsubscribe-auth/ABcq67tjeLuTKdqyAOqxrnTfqPFUFpeAks5rT466gaJpZM4LnmYK .

yan12125 commented 7 years ago

Which country? I got a proxy from United States and it doesn't work.

marafa commented 7 years ago

United States is the country

yan12125 commented 7 years ago

Now I got 404 with various US proxies. Is the video still alive?

marafa commented 7 years ago

Yes it is. It's not due to be switched out till the end of the month

On Jan 20, 2017 10:47 AM, "Yen Chi Hsuan" notifications@github.com wrote:

Now I got 404 with various US proxies. Is the video still alive?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rg3/youtube-dl/issues/11775#issuecomment-274104228, or mute the thread https://github.com/notifications/unsubscribe-auth/ABcq6xdzGNutTinzELU8c6M1g0T5r0Aqks5rUNbhgaJpZM4LnmYK .

zoocey commented 7 years ago

My other report was closed #10419. I am not in the US but crackle is available in my region (with a limited selection). If it is available in your region any video on the crackle page should reproduce the issue. I can also watch American videos using a paid VPN with no issue.

As a work around if you have a way to sniff traffic when playing a video, e.g. an adblocker (ublock origin) or experience with your browsers developer tools. Filter for json. For your video you should see a url like ..Service.svc/details/media/2492959/US?format=json

Append that number to your url http://www.crackle.com/7-seconds/2492959 and it will work with youtube-dl.

ajp69 commented 7 years ago

Should download if you use the correct link

http://www.crackle.com/7-seconds/2492959

Shows a 1080p video at 5052k and aac audio at 128kbps

Also it is easier to get the correct link without sniffing the traffic. When you have the video open in the browser just click the video under the you might also like section should re open with the correct link.

marafa commented 7 years ago

can someone attach a copy of the crackle.com/7-seconds page? it might help

oxguy3 commented 7 years ago

PSA: It's very easy to get the correct URL for a movie without having to do any fancy traffic sniffing. Just scroll down to where it says "NOW PLAYING" and click on the movie poster. You'll then be taken to the URL with the number in it. Here's a screenshot; I've circled the link you should click:

oxguy3 commented 7 years ago

By the way youtube-dl developers, here's how you can add support for the number-less URLs: For all URLs of the format "http://www.crackle.com/SOME-SLUG", make a query to https://web-api-us.crackle.com/Service.svc/details/channel/SOME-SLUG/US?format=json (side note: you'll have to change the country code at the end from "US" to whatever country the user's IP address is in). Then, get the attribute "ID" of the object "FeaturedMedia" from that JSON response, and append that to the URL.

If that endpoint returns an empty response, that means the URL was invalid and you should throw an "unsupported URL" error message. If the endpoint returns a valid JSON response, but the "FeaturedMedia" key doesn't exist, that probably means the URL was a TV show rather than a movie, and you should throw an "unsupported URL" error message.

Here are examples of each of these cases:

yan12125 commented 7 years ago

Does crackle support countries other than United Stated? https://web-api-us.crackle.com/Service.svc/details/channel/7-seconds/US?format=json returns valid results yet https://web-api-us.crackle.com/Service.svc/details/channel/7-seconds/TW?format=json gives HTTP 401. If only US is possible I'd like to hardcode it - detecting users' geolocation is quite difficult.

oxguy3 commented 7 years ago

It definitely works in a couple other countries, but the selection of movies/shows is different (looks like 7 Seconds happens to be US-only). Here's the endpoint to retrieve the list of all movies for a region: https://web-api-us.crackle.com/Service.svc/browse/movies/full/all/alpha-asc/US/20/1?format=json And here's the endpoint for getting all shows: https://web-api-us.crackle.com/Service.svc/browse/shows/full/all/alpha-asc/US/20/1?format=json

If you plug CA or AU or PR in place of US in those URL, you still get results. I found an endpoint that retrieves the list of all valid regions: https://web-api-us.crackle.com/Service.svc/appconfig?format=json

Note that some regions actually require you use a different server -- web-api-es.crackle.com for Spanish-speaking countries or web-api-br.crackle.com for Brazil (this info is included in that appconfig API response). Also, it seems that some of the regions are very sparse -- it looks like Mexico has 3 shows and 0 movies, for example (but not all non-US regions are like this -- Canada and Australia both have a lot of content).