search

codemanki / cloudscraper

--DEPRECATED -- 🛑 🛑 Node.js library to bypass cloudflare's anti-ddos page
MIT License
599 stars 139 forks source link

Replace request with node-fetch #326

Open minas90 opened 4 years ago

minas90 commented 4 years ago

Hi there, I think it's a good idea to replace request, as it's deprecated now.

node-fetch seems to be the best future proof option at the moment.

kevinvanrijn commented 4 years ago

A rewrite of cloudscraper might take a while seeing how little activity this repo has had, so another short term solution would be to use a maintained fork of request during the grace period.

https://github.com/postmanlabs/postman-request seems to be the only well maintained fork currently.

Agreed on node-fetch as a replacement fwiw.

RedNinjaX commented 4 years ago

At this rate we may need a cloudscraper replacement as well

codemanki commented 4 years ago

@kevinvanrijn @RedNinjaX you are always welcome to make a PR to this repo and help others and me with the rate.

kevinvanrijn commented 4 years ago

I can't promise anything for the rewrite side of things (I simply don't have the time for any significant contributions) but I will be looking to move my own request-dependent projects over to a maintained fork.

So if you don't see cloudscraper moving away from request anytime soon then I can make the relevant changes here as well.

I'll first be waiting for the dust to settle though. I want to be sure no other forks pop up and see if the Postman guys are still interested in maintaining their fork.

kevinvanrijn commented 4 years ago

I've received confirmation that postman-request is alive. https://github.com/request/request/issues/3142#issuecomment-587569671

In addition, a new release has been created since the deprecation of request. So it looks like postman-request will continue to be maintained for the foreseeable future.

I'll leave the next steps up to you @codemanki , would you be interested in having cloudscraper moved over to postman-request or is a rewrite coming sooner than expected?

codemanki commented 4 years ago

@kevinvanrijn what would be the benefits of replacing request with postman-request as of now? Cloudscraper doesn't use a lot of request features, so ( imho ) there are no critical issues directly related to outdated request library.

minas90 commented 4 years ago

@codemanki The main reason for deprecating request is the fact that it is not in line with modern js. And at some point it may stop working on a newer version of node after some breaking changes.

My main concern is the fact that in the past every time I ran npm install I got exactly 1 warning from dozen or so deps I use and it came from cloudscraper. I use Node.js v10 or later, but I get nagged to install Brotli and even for older versions brotli is optional I think, right?

And a week ago I started to get 2 warnings, brotli and request and both come from cloudscraper and it's kind of annoying tbh. My automated deployment script does npm install and I deploy dozens of times per day. Every time these 2 warnings grab my attention. Of course I can simply tweak the script to hide warnings, but what if there is something else more critical?

kevinvanrijn commented 4 years ago

@codemanki It would mainly be to not depend on a deprecated package. It's not just request that isn't getting updates, all of it's dependencies will also never be updated beyond what the current package.json defines. A security issue is bound to happen eventually.

I don't think any of the bugfixes made in postman-request are relevant to cloudscraper but it does fix some edge-cases in request's behavior that may be relevant to cloudscraper users.

If you'd prefer to not make any decisions yet that's also fine with me.