search

KJHJason / Cultured-Downloader

A project to automate the process of downloading images and other attachment files from platforms like Fantia and more!
GNU General Public License v3.0
72 stars 7 forks source link

[Bug]: Cloudflare Challenge Page on Pixiv Fanbox #259

Closed gentle-hen closed 4 months ago

gentle-hen commented 4 months ago

Which program are you seeing this problem on?

Cultured Downloader (Go)

Program Version

5.1.0

Operating System

Windows 10

What error code did you receive? (If any)

1003 (Input Error)

Steps to reproduce the bug

  1. Go to "Sessions"
  2. Enter in my fanboxsessid
  3. "Error 1003: Pixiv Fanbox cookie is invalid" appears after clicking "Save"

Expected behaviour

I expected my fanboxsessid to work as I can see the paid content on chrome without problem.

Relevant log output (If any)

No response

Screenshots (Optional)

No response

KJHJason commented 4 months ago

Looks like Pixiv Fanbox has enabled Cloudflare under attack mode.

Will take more time to test if using chromedp or go-rod will work...

KJHJason commented 4 months ago

You can refer to the workarounds found in https://github.com/Nandaka/PixivUtil2/issues/1342 such as using a VPN to JP which should address the Cloudflare challenge page issue for now.

Not sure if this is temporary or permanent but probably be testing chromedp if it goes through the challenge page. Otherwise, I will have to add new inputs for Pixiv Fanbox for the Cloudflare clearance cookie.

However, with the hacky Cloudflare clearance cookie workaround, the __cf_bm cookie has a max-age of 15 minutes and is expected to only last for an hour so it's not an ideal UX for the user to re-enter the cookie value every hour.

KJHJason commented 4 months ago

Update: using go-rod with stealth works, but it uses the leakless binary to kill the automated browser which is commonly used by malware and gets wrongly identified by AV like bitdefender.

Edit: nvm, not sure what happened but now the challenge page is popping up.

KJHJason commented 4 months ago

However, with the hacky Cloudflare clearance cookie workaround, the __cf_bm cookie has a max-age of 15 minutes and is expected to only last for an hour so it's not an ideal UX for the user to re-enter the cookie value every hour.

How do you even do this workaround? I apparently couldn't properly parse the technical lingo used by the ones who discussed it in that thread, because my attempts to implement it aren't working. I even downloaded a zip of the master version and unpacked it.

Hello @MarqFJA87, I have not used PixivUtils2 but you need the cf_clearance and __cf_bm cookies that you can obtain from your browser. You can refer to https://github.com/Nandaka/PixivUtil2/issues/1342#issuecomment-2198376006.

Edit: Looks like you deleted your original comment

KJHJason commented 4 months ago

Update: using go-rod with stealth works, but it uses the leakless binary to kill the automated browser which is commonly used by malware and gets wrongly identified by AV like bitdefender.

Edit: nvm, not sure what happened but now the challenge page is popping up.

Managed to get it working via https://github.com/KJHJason/Cultured-Downloader-Logic/commit/6e1997b8eb3de33b240e86df0ad43c8668f9fe7f though I'm not sure if it's consistent...

More testings will be required.

MarqFJA87 commented 4 months ago

Hello @MarqFJA87, I have not used PixivUtils2 but you need the cf_clearance and __cf_bm cookies that you can obtain from your browser. You can refer to Nandaka/PixivUtil2#1342 (comment).

Edit: Looks like you deleted your original comment

Yeah, I just realized that the link I had clicked had taken me outside the PixivUtil2 repository and into another one.

And I did manage to find those cookies, but apparently I need something else to get it working, because I still keep getting the same error. Changing my user-agent just runs into an older issue with Pixiv that causes a login failure, which others have reported while trying the suggested fixes.

gentle-hen commented 4 months ago

Update: using go-rod with stealth works, but it uses the leakless binary to kill the automated browser which is commonly used by malware and gets wrongly identified by AV like bitdefender. Edit: nvm, not sure what happened but now the challenge page is popping up.

Managed to get it working via KJHJason/Cultured-Downloader-Logic@6e1997b though I'm not sure if it's consistent...

More testings will be required.

I'm sorry, but how do I use KJHJason/Cultured-Downloader-Logic@6e1997b to get it working? By the way, I tried using a VPN to switch to JP but did not work. Also, I forgot to ask in the initial post, is the executable version of your program the Python version?

Thank you.

KJHJason commented 4 months ago

Version 5 and above uses Go now with a GUI.

If you're using a CLI based program, it's the old Python program tho has been reported to no longer work due to webdriver manager.

With regards to VPN workaround, I will try test on my end first. Otherwise I will speed up the development of V5.2.0 👍

KJHJason commented 4 months ago

Hello @gentle-hen, it seems like Pixiv Fanbox has relaxed their Challenge Page security level. Hence, you should be able to set your Fanbox Session ID.

However, I will still work on implementing a Cloudflare solver into the program in the event this happens again 👍

gentle-hen commented 4 months ago

Hello @gentle-hen, it seems like Pixiv Fanbox has relaxed their Challenge Page security level. Hence, you should be able to set your Fanbox Session ID.

However, I will still work on implementing a Cloudflare solver into the program in the event this happens again 👍

That's great to hear! Do you want me to close this, or keep it open?

Thank you for all the work!

KJHJason commented 4 months ago

Just leave it open, thanks!