e621 Ripper is broke - pls update

[TristanNachgemacht]

Hey this: e621.net had a big update recently and since then the ripper doesn't work anymore. A patch would make me happy. Thanks in advance.

• Ripme version: 1.7.91
• Java version: cant find it sorry qwq
• Operating system: Windows 10 Home x64 (10.0.18363 Build 18363)

[viktor02]

e621 changed url from https://e621.net/post/index/1/search to https://e621.net/posts?tags=search&page=1

[TristanNachgemacht]

That's supposed to mean something? It's not working for me ;(

[wickedwzrd]

Even with manually editing the url to the old format doesn't work. There are images under this search term yet it says nothing has been found.

[hellsunicorn]

I concur with the others. This issue desperately needs patching.

[TristanNachgemacht]

I concur with the others. This issue desperately needs patching.

Yeah it would be very nice D:

[JotaRandom]

So far every single ripper or tracker related to e621 is broken, this since e621 update they site.

I think they bróke all the trackers too.

[blackbirddx]

I made a fix, try it if you want: https://github.com/blackbirddx/ripme This is compiled jar (if you don't trust me, check my repo and compile it yourself): ripme-e621.zip Please tell me if you encounter any problems, so I can fix them before making pull request.

[viktor02]

I made a fix, try it if you want: https://github.com/blackbirddx/ripme This is compiled jar (if you don't trust me, check my repo and compile it yourself): ripme-e621.zip Please tell me if you encounter any problems, so I can fix them before making pull request.

I can't create issue in your repo, but i got an error.

imgs

[Imgur](https://i.imgur.com/1fh6KoC.png) with captcha in url [Imgur](https://i.imgur.com/liU1yiI.png) no captcha

[blackbirddx]

Fixed, here's binary: ripme-e621-fix-2.zip

Btw. how did you get the url with captcha?

[viktor02]

Fixed, here's binary: ripme-e621-fix-2.zip

Btw. how did you get the url with captcha?

Sometimes it gives a cloudflare page when I connect via VPN. I just copy the full url after "human check"

[viktor02]

Fixed, here's binary: ripme-e621-fix-2.zip

Btw. how did you get the url with captcha?

I still get the 403 error. Is it a problem on my side?

[blackbirddx]

Fixed, here's binary: ripme-e621-fix-2.zip Btw. how did you get the url with captcha?

I still get the 403 error. Is it a problem on my side?

It may be problem with the vpn and cloudflare. As the app suggests, including cookies might solve it, but I never got the cloudflare captcha, so I don't really know what to do about it. Maybe try some other vpn (or connecting without vpn).

[viktor02]

It may be problem with the vpn and cloudflare. As the app suggests, including cookies might solve it, but I never got the cloudflare captcha, so I don't really know what to do about it. Maybe try some other vpn (or connecting without vpn).

The problem is that I connected with and without vpn. And in any case, I get an error.

[blackbirddx]

The problem is that I connected with and without vpn. And in any case, I get an error.

Before the e621 update, were you also getting the captcha? And did the ripper work for you then?

[viktor02]

The problem is that I connected with and without vpn. And in any case, I get an error.

Before the e621 update, were you also getting the captcha? And did the ripper work for you then?

I didn't have a captcha before the update. But I didn't use the ripper until now

[blackbirddx]

I didn't have a captcha before the update. But I didn't use the ripper until now

I'll try to find a solution for that. I've been trying to get a captcha by using a couple free vpns, which I hoped would trigger cloudflare protection, but so far it didn't happen. I'll look into couple more and maybe I'll get lucky.

[viktor02]

I'll try to find a solution for that

Try to log in with incognito. I have so without vpn protection CloudFlare triggers. As you come in with vpn - it is not. The problem with 403 is possible due to the blocking of the e621 by provider in Russia. However, error 403 remains even with vpn.

[TristanNachgemacht]

Ooof i hate it so much scince the update -__-

[TristanNachgemacht]

Fixed, here's binary: ripme-e621-fix-2.zip

Btw. how did you get the url with captcha?

There is still the blacklist problem, I cannot download images that are blacklisted for standard users.

[blackbirddx]

There is still the blacklist problem, I cannot download images that are blacklisted for standard users.

They are blacklisted for a reason. Anyway, login cookies might be a solution here, I'll look into it when I find some time. Although it seems like a bit of an overkill to me. Or I could use e621 API to download everything, but then some users might get stuff they don't want to see.

[blackbirddx]

Try to log in with incognito. I have so without vpn protection CloudFlare triggers. As you come in with vpn - it is not. The problem with 403 is possible due to the blocking of the e621 by provider in Russia. However, error 403 remains even with vpn.

I finaly got the captcha - had to install Windows to get some russian vpn working (just couldn̈́t do it on Linux) and I think cookies might really solve it. I'll try to code it and will report if it works.

They are blacklisted for a reason. Anyway, login cookies might be a solution here, I'll look into it when I find some time. Although it seems like a bit of an overkill to me. Or I could use e621 API to download everything, but then some users might get stuff they don't want to see.

I think I will just add option to config that would enable the global blacklist bypass (would still be disabled by default).

[viktor02]

I finaly got the captcha - had to install Windows to get some russian vpn working (just couldn̈́t do it on Linux) and I think cookies might really solve it. I'll try to code it and will report if it works.

Lol, thank you, i will follow your repo

[TristanNachgemacht]

Try to log in with incognito. I have so without vpn protection CloudFlare triggers. As you come in with vpn - it is not. The problem with 403 is possible due to the blocking of the e621 by provider in Russia. However, error 403 remains even with vpn.

I finaly got the captcha - had to install Windows to get some russian vpn working (just couldn̈́t do it on Linux) and I think cookies might really solve it. I'll try to code it and will report if it works.

They are blacklisted for a reason. Anyway, login cookies might be a solution here, I'll look into it when I find some time. Although it seems like a bit of an overkill to me. Or I could use e621 API to download everything, but then some users might get stuff they don't want to see.

I think I will just add option to config that would enable the global blacklist bypass (would still be disabled by default).

It would be very great of you if you could install such a blacklist bypass. Because it's starting to annoy me.

[blackbirddx]

So I think I've finally done it. Compiled jar as always: ripme-e621fix.zip

Both of the fixes require setting things in config file: https://github.com/RipMeApp/ripme/wiki/Config-options The config can only be changed when the app is not running. You also need to find your cookies for e621. I use Cookie Quick Manager in Firefox.

Try to log in with incognito. I have so without vpn protection CloudFlare triggers. As you come in with vpn - it is not. The problem with 403 is possible due to the blocking of the e621 by provider in Russia. However, error 403 remains even with vpn.

You need to complete the captcha challenge in your browser. After that, two cookies will be set for e621 - __cfduid and cf_clearance. In the config file (rip.properties) create new option e621.cookies and set it to: __cfduid=(content of the cookie);cf_clearance=(content of the cookie) (not entirely sure if the cfduid is strictly necessary, but I rather included it)

Then you need to find out what your browser's user agent is. Just google "what is my user agent". In config create option e621.useragent and set it to your ua.

There is still the blacklist problem, I cannot download images that are blacklisted for standard users.

At the end I decided against the api method, because technical limitations and me being lazy :) Instead, because I already had cookies for the last fix, I used those.

First you need to log in with an account and have Remember ticked. Then new cookie named remember is set for e621, so copy its content. In options create new option e621.cookies and set it to remember=(content of the cookie)

Then it should stop using the global anonymous user blacklist. Another advantage of being logged in is that it will use your own blacklist, so if you don't want to see something, you can block it. It also allows wildcards (*) in tags.

---

You can of course combine the two fixes, just set your e621.cookies to: __cfduid=(cookie);cf_clearance=(cookie);remember=(cookie)

Please note, that the cookies have limited lifespan, so eventually you will again have to find your cookies and set them in the config (the captcha I think only lasts couple of hours, the account around a week?)

[viktor02]

Both of the fixes require setting things in config file:

I did as you asked - added a new parameters and saved it. However, this did not work, and importantly, the new parameters was erased from the file.

[blackbirddx]

Both of the fixes require setting things in config file:

I did as you asked - added a new parameters and saved it. However, this did not work, and importantly, the new parameters was erased from the file.

You need to do it when the app is closed, then save and run the app

[hellsunicorn]

Just tried your latest fix and it worked for me. Thanks a lot, blackbirddx

[viktor02]

You need to do it when the app is closed, then save and run the app

Ok, Apparently that time I was doing something wrong. In any case, it didn’t work for me. I was passed captcha less than 5 minutes ago

 

[blackbirddx]

You need to do it when the app is closed, then save and run the app

Ok, Apparently that time I was doing something wrong. In any case, it didn’t work for me. I was passed captcha less than 5 minutes ago

It's e621.cookies, not e621.cookie.

[TristanNachgemacht]

I'm a bit confused right now, I've downloaded version 1.7.92 and still the blacklist is still used, how do I set it to ignore it?

[viktor02]

It's e621.cookies, not e621.cookie.

thank you very much, it works

[blackbirddx]

I'm a bit confused right now, I've downloaded version 1.7.92 and still the blacklist is still used, how do I set it to ignore it?

If you downloaded the official build, my patch is still not part of it. You need to download my version.

Then follow the steps I described a few comments ago.

[blackbirddx]

I'm a bit confused right now, I've downloaded version 1.7.92 and still the blacklist is still used, how do I set it to ignore it?

Please let me know if it works for you. I want to make pull request to get this fix into the app, but I first need to know if the patch works.

[TristanNachgemacht]

I'm a bit confused right now, I've downloaded version 1.7.92 and still the blacklist is still used, how do I set it to ignore it?

If you downloaded the official build, my patch is still not part of it. You need to download my version.

Then follow the steps I described a few comments ago.

I just don't understand the steps, can you make me a config or show me exactly where I have to install it with the help of screenshots? That would be really very nice.

[blackbirddx]

I just don't understand the steps, can you make me a config or show me exactly where I have to install it with the help of screenshots? That would be really very nice.

e621-ripme-fix-howto.pdf

[TristanNachgemacht]

I just don't understand the steps, can you make me a config or show me exactly where I have to install it with the help of screenshots? That would be really very nice.

e621-ripme-fix-howto.pdf

OMG THX YOU SO MUCH, ITS AMAZING :DD IT WORKS PERFECTLY :+1: :100: :1st_place_medal:

[blackbirddx]

You're all welcome :) Hope the pull request gets merged soon, so it may get into the next release.