JimmyLaurent / cloudflare-scraper

A package to bypass Cloudflare's protection
MIT License
286 stars 30 forks source link

Reduce cpu usage #11

Open ElijahDD opened 4 years ago

ElijahDD commented 4 years ago

It possible to reduce cpu usage? When i launch script in some threads on different sites - it usage 8/16 core/threads near on 80%

JimmyLaurent commented 4 years ago

Don't forget that it's an instance of chrome that you're launching here, each time you're opening a new website.

Apart maybe from tweaking some arguments on the launch of the chrome instance or maybe sharing the same browser instance and using multiple tabs, I don't see how it's possible.

If you find some nice tweaks, I'd be happy to merge it in the codebase.

ElijahDD commented 4 years ago

Don't forget that it's an instance of chrome that you're launching here, each time you're opening a new website.

Apart maybe from tweaking some arguments on the launch of the chrome instance or maybe sharing the same browser instance and using multiple tabs, I don't see how it's possible.

If you find some nice tweaks, I'd be happy to merge it in the codebase.

How i can change arguments for chrome? Want to try this:

      '--disable-dev-shm-usage',
      '--disable-accelerated-2d-canvas',
      '--no-first-run',
      '--no-zygote',
      '--single-process', // <- this one doesn't works in Windows
      '--disable-gpu',
      '--hide-scrollbars',
      '--mute-audio',
      '--disable-gl-drawing-for-tests',
      '--disable-canvas-aa',
      '--disable-2d-canvas-clip-aa',
      '--use-gl=swiftshader', // better cpu usage with --use-gl=desktop rather than --use-gl=swiftshader, still needs more testing.
      '--enable-webgl',
      '--no-first-run',
      '--disable-infobars',
      '--disable-breakpad',
JimmyLaurent commented 4 years ago

You’ll have to monkey patch it in ‘createBrowser.js’.

There’s a list a args, just add yours.

Webuser6666 commented 4 years ago

You’ll have to monkey patch it in ‘createBrowser.js’.

There’s a list a args, just add yours.

Hello, why - TypeError: browser.setMaxListeners is not a function ?