Add support to gfycat.com Links that redirect to gifdeliverynetwork.com now & redgifs.com

[Anon1337Elite]

Is your feature request related to a problem? Please describe.

Most gfycat links now redirect to gifdeliverynetwork. This makes RMD to error out and not download the file. The path is the same the only change is the gfycat to gifdeliverynetwork.

Describe the solution you'd like:

Add support for gifdeliverynetwork (gfycat links that redirect to it) & redgifs.com

[JustCampMan]

I'm pretty sure shadowmoose can't do anything about it as gfycat is handled by youtube-dl.

You could raise this issue on their github.

[shadowmoose]

I've looked at this, and their new site is missing an API currently. Given that there is no clean way to reliably extract (without using some page-scraping hack), I will likely not support this until either YTDL updates to handle it themselves, or until gfycat adds the missing API to their new site.

[Anon1337Elite]

I've looked at this, and their new site is missing an API currently. Given that there is no clean way to reliably extract (without using some page-scraping hack), I will likely not support this until either YTDL updates to handle it themselves, or until gfycat adds the missing API to their new site.

I found this i don't know how much help it is. But this guy is saying that he is calling their api for redgifs too. I don't understand it, so i will just leave it here:

https://github.com/RipMeApp/ripme/pull/1672

[ItsPizzaTime1501]

A quick fix I found for this is to add 'force_generic_extractor': True to ytdl_opts if the link contains gfycat.com, I tested it on 10 different links and they all seemed to work. The generic extractor should also work for redgifs

[TesticularMass]

A quick fix I found for this is to add 'force_generic_extractor': True to ytdl_opts if the link contains gfycat.com, I tested it on 10 different links and they all seemed to work. The generic extractor should also work for redgifs

I still found that a lot of links just weren't being downloaded. For some reason, gfycat was disconnecting from me and ytdl would not attempt the default 10 attempts. I found that also adding 'retries': 10 to ydl_opts made it work. Now all gfycat, gifdeliverynetwork, and redgifs downloads work.

[shadowmoose]

Hey, just checking back in - until there is some official support (or added support via YTDL) I will not be able to add the new site. Maintaining the code to scrape from their rapidly-evolving website is outside RMD's scope.

[parasiteoflife]

@ItsPizzaTime1501 @Nosidak can any of you guys share a Windows build with those modifications?

[Anon1337Elite]

Hey, just checking back in - until there is some official support (or added support via YTDL) I will not be able to add the new site. Maintaining the code to scrape from their rapidly-evolving website is outside RMD's scope.

@shadowmoose This is still an issue. Heard you doing a rewrite, could this possible be fixed in the future, from RMD Side ?

Redgifs has been fixed by YTDL (Still broken in RMD), But gifdeliverynetwork is still broken all around. YTDL Downloads the video if you give it a direct gifdeliverynetwork link, but it doesn't if you give it a gfycat link that redirects to gifdeliverynetwork. Something that might help the issue is that "https://gfycat.com/XXX" redirects to "https://www.gifdeliverynetwork.com/XXX" and if it is replaced by "https://www.redgifs.com/watch/XXX" , the link/video will still work on the redgifs/gifdeliverynetwork domain.

Maybe something can be implemented of sorts of replacing "https://gfycat.com/" with "https://www.redgifs.com/watch/" if link fails as a band aid fix ?

[shadowmoose]

That's a good catch. If YTDL is supporting the direct site links now, I can adjust the new download handlers to work around the redirect.

[Anon1337Elite]

That's a good catch. If YTDL is supporting the direct site links now, I can adjust the new download handlers to work around the redirect.

Just tested it again to make sure. Direct link works fine. The redirect is the only thing that screws it up.

Tested the same gif with both https://www.gifdeliverynetwork.com/XXX & https://www.redgifs.com/watch/XXX.

D:\RMD\Newfolder>youtube-dl.exe -a, --batch-file D:\RMD\Newfolder\new.txt -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-a,', '--batch-file', 'D:\\RMD\\Newfolder\\new.txt', '-v']
[debug] Batch file urls: ['https://www.redgifs.com/watch/XXXX']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.03.03
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: none
[debug] Proxy map: {}
[generic] XXXX: Requesting header
WARNING: Falling back on generic information extractor.
[generic] XXXX: Downloading webpage
[generic] XXXX: Extracting information
[download] Downloading playlist: XXXX by unclebob | RedGIFs
[generic] playlist XXXX by unclebob | RedGIFs: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://thumbs2.redgifs.com/XXXX-mobile.mp4'
[download] Destination: XXXX by unclebob _ RedGIFs-XXXX.mp4
[download] 100% of 11.16MiB in 00:00
[download] Finished downloading playlist: XXXX by unclebob | RedGIFs

D:\RMD\Newfolder>youtube-dl.exe -a, --batch-file D:\RMD\Newfolder\new.txt -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-a,', '--batch-file', 'D:\\RMD\\Newfolder\\new.txt', '-v']
[debug] Batch file urls: ['https://www.gifdeliverynetwork.com/XXXX']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2021.03.03
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: none
[debug] Proxy map: {}
[generic] XXXX: Requesting header
WARNING: Falling back on generic information extractor.
[generic] XXXX: Downloading webpage
[generic] XXXX: Extracting information
[download] Downloading playlist: XXXX by unclebob
[generic] playlist XXXX by unclebob: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://thumbs2.redgifs.com/XXXX.mp4'
[download] Destination: XXXX by unclebob-XXXX.mp4
[download] 100% of 11.16MiB in 00:00
[download] Finished downloading playlist: XXXX by unclebob

[tampa-bay]

I tested about 100 older gfycats that redirect, and played with ytdl source to make it work, at least in those cases. In extractor/gfycat.py Lines 65-69, I changed:

65 video_id = self._match_id(url)
66
67 gfy = self._download_json(
68 'https://api.gfycat.com/v1/gfycats/%s' % video_id,
69 video_id, 'Downloading video info')

to:

65 video_id = self._match_id(url).lower()
66
67 gfy = self._download_json(
68 'https://api.gifdeliverynetwork.com/v1/gfycats/%s' % video_id,
69 video_id, 'Downloading video info')

Don't know if this will help or not.

[shadowmoose]

@tampa-bay While this problem is fixed in the typescript rewrite, the fact that there is a matching cross-site API is useful information. Unfortunately, it seems like the gifdeliverynetwork api does not correctly return valid gfycat links, so it will not work as a 1:1 replacement.

I'm open to PRs if anybody wants to patch this functionality in, otherwise this is already solved in the rebuild.