search

sayem314 / hooman

http interceptor to hoomanize cloudflare requests
https://www.npmjs.com/package/hooman
MIT License
148 stars 18 forks source link

Reduce dependency overhead #17

Open ghost opened 4 years ago

ghost commented 4 years ago

jsdom and user-agents bloat this package to 6.2 MB minified. I suggest making the user agent user supplied and replacing jsdom with cheerio or something like this.

sayem314 commented 4 years ago

JSDOM can't be replaced with Cheerio. The goal of JSDOM is to provide an identical DOM environment as what we see in the browser. And about user-agents, yes I might remove this in the future.

ghost commented 4 years ago

Thanks for the answer. Is jsdom actually used for anything but parsing the html? I didn't see this when checking the code.

sayem314 commented 4 years ago

It's used here both for retrieving form data and solving challenges from Cloudflare.

https://github.com/sayem314/hooman/blob/fd80462e3258a127199c1d4066a1f1bf4ff4efab/lib/core.js#L11-L33

Kikobeats commented 4 years ago

@sayem314 can I suggest you use top-user-agents?

It's just a JSON of user agents, updated every time you fetch the package.

849 bytes of package 🙂