search

4pr0n / ripme

Downloads albums in bulk
MIT License
912 stars 205 forks source link

Gfycat 403 error #155

Open ghost opened 9 years ago

ghost commented 9 years ago

I just downloaded the new version, but I'm still having gfycat issues. On here and Gonewilder.

On RipMe, this is what comes up:

2015-01-10 18:52:02,719 147259580 [Thread-5234] DEBUG utils.RipUtils - Checking http://gfycat.com/NeglectedMellowKoi 2015-01-10 18:52:02,719 147259580 [Thread-5234] INFO ripper.AbstractRipper - Retrieving http://gfycat.com/NeglectedMellowKoi 2015-01-10 18:52:02,845 147259706 [Thread-5234] WARN ripper.AbstractRipper - Error while loading http://gfycat.com/NeglectedMellowKoi org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=http://gfycat.com/NeglectedMellowKoi at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:449) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:424) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:178) at com.rarchives.ripme.utils.Http.response(Http.java:130) at com.rarchives.ripme.utils.Http.get(Http.java:111) at com.rarchives.ripme.ripper.rippers.video.GfycatRipper.getVideoURL(GfycatRipper.java:67) at com.rarchives.ripme.utils.RipUtils.getFilesFromURL(RipUtils.java:45) at com.rarchives.ripme.ripper.rippers.RedditRipper.handleURL(RedditRipper.java:186) at com.rarchives.ripme.ripper.rippers.RedditRipper.parseJsonChild(RedditRipper.java:153) at com.rarchives.ripme.ripper.rippers.RedditRipper.getAndParseAndReturnNext(RedditRipper.java:86) at com.rarchives.ripme.ripper.rippers.RedditRipper.rip(RedditRipper.java:61) at com.rarchives.ripme.ripper.AbstractRipper.run(AbstractRipper.java:316) at java.lang.Thread.run(Unknown Source)

Sometimes, I get the gfy, most of the time not. I generally get the 403 error. I've tried switching to a different network via VPN, and even switching computers. I still get 403 errors.

I searched around, and it seems like it's a common issue (?), but I'm not savvy enough to know if it's on my end or not.

Thanks!

4pr0n commented 9 years ago

Kind of reproduced by ripping /r/nsfw_html5

I updated the user agent & now send the referrer when ripping gfycat links from reddit.

Haven't gotten a 403 since the change, so... maybe that fixes it?

Some people on /r/gfycat mentioned that these errors just suddenly started happening [1]. Maybe it's due to server load balancing & ignoring automated requests?

4pr0n commented 9 years ago

Hoping this fixes it.

ghost commented 9 years ago

So far, no 403 errors. I'm kind of busy right now, but I will check again in the next few days and test out some users with gfys to see if I get any 403s.

I know this is for RipMe, but any idea why 403 errors coming up for Gonewilder as well? It's also having trouble with vidme videos. Downloading very slow or not at all. I'm wondering if it's similar issues that seem to have just been fixed in the latest RipMe patch you committed.

ghost commented 9 years ago

So after a bit more testing it seems to work... and also that commit you also made to Gonewilder seems to have worked as well for gfycat 403 issue. The only thing that it says (for Gonewilder) is this:

[2015-01-17T05:24:30Z] ImageUtils: gfycat.com: getting http://www.gfycat.com/BabyishGroundedBoutu [2015-01-17T05:24:32Z] Gonewild: ShesAWitch0: process_url: downloading #1 https://zippy.gfycat.com/BabyishGroundedBoutu.webm [2015-01-17T05:24:40Z] Gonewild: ShesAWitch0: process_url: #1 cannot identify image file

But I think that has to do with the thumbnail.

I will try to test more later on.

Thanks so much!