SpangleLabs
commented
3 years ago Yup Tonnes and tonnes of issues about it not working, and no update from the owner in many months. Pretty sure it's dead. Some folks moved over to https://github.com/VeNoMouS/cloudscraper/ but I believe that's moved over to a subscription model where you have to contact the owner over discord? and the owner has disabled issues, and shuts down any comments on PRs about functionality.
I've not seen a lot of other options around.
- Some folks have said to use scrapy-splash: https://github.com/scrapy-plugins/scrapy-splash/issues/92#issuecomment-680050082 but I'm not sure how that goes.
- There's some JS options around, again, not sure whether they're working:
- https://www.npmjs.com/package/hooman (last update 5 months ago, repo issues say it's broken, dev says he'll work on it, but is very busy https://github.com/sayem314/hooman/issues/34#issuecomment-703052304)
- https://www.npmjs.com/package/cloudflare-scraper (last update 6 months ago, repo issues say it's broken)
- https://www.npmjs.com/package/@ctrl/cloudflare (last update 8 months ago, repo archived, suggests )
- https://www.npmjs.com/package/cloudscraper (last update a year ago)
- Browser based solutions:
That last one sounds like the best bet to me. It's a docker image that spins up browsers to do the request, so it's going to be slow and quite memory-heavy. But development seems active, latest commit to master was 5 days ago. But I've not tried it myself.
VeNoMouS
Is this module dead? Owner not supporting anymore?