search

Xonshiz / comic-dl

Comic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.
https://github.com/Xonshiz/comic-dl
MIT License
545 stars 68 forks source link

[Comic-dl] Auto processing with error for readcomiconline.to Detected the new Cloudflare challenge. #271

Closed jloffredo closed 3 years ago

jloffredo commented 4 years ago

Getting that error more and more.

[Comic-dl] Auto processing with error for https://readcomiconline.to/Comic/[comicurl] : Detected the new Cloudflare challenge.

magnuschase293 commented 4 years ago

Same

vengeance101 commented 4 years ago

After digging a bit, I assume this is why I'm getting a 503. cloudscraper seems to be shut down (at least, no repo anymore). I manually ran cloudscraper against readcomiconline and it wouldn't bypass (always returned cloudfare page).

I think this project is just broken until someone can bypass the anti-scrapers again.

magnuschase293 commented 4 years ago

I've been looking into the matter myself and thought a possibility may be to use cfscrape instead of cloudscraper (didn't tried It). Also I found out you can simply fully load the comic and save the webpage which downloads all the .jpgs inside a folder.

vengeance101 commented 4 years ago

I think selenium would work pretty well here as well, but I'm sure another cloudfare workaround will come out soon.

magnuschase293 commented 4 years ago

Thanks, I will try to use Selenium, but I need to learn the module from scratch. Do you know if it can manage the ChromeDevTools protocol??? It's for another project.

vengeance101 commented 4 years ago

Yes, fairly certain it can. I haven't used it much either but if it provides a workaround to this I might just make a small little script.

edit: https://medium.com/@ohanaadi/chrome-devtools-and-selenium-4-eadab5d755b7

magnuschase293 commented 4 years ago

I actually made the script if you are interested here's the repo

Peepijeep commented 4 years ago

Hello, I dont know how to use that script. Can you build a windows-binary please, so I can use it? I really want to download the comics from comic.dl.

Xonshiz commented 3 years ago

I was using selenium initially for readcomingonline website, but selenium is heavy and slow. I think the issue is fixed now that we switch the CF challenge library. I'm closing this for now, please feel free to re-open if the issue is still there in the latest binary.

Don't forget to run the script with the verbose flag and share the generated error log as well.