maaaaz / webscreenshot

A simple script to screenshot a list of websites
GNU Lesser General Public License v3.0
653 stars 162 forks source link

No option to wait until page is loaded #46

Open kylefrost opened 4 years ago

kylefrost commented 4 years ago

I am trying to screenshot a webpage that takes a few moments to fully load, and the script is simply running and screenshotting before the page can load. Is there an option to set a sleep period? Or could one be added?

maaaaz commented 4 years ago

Hello, try to play with --ajax-max-timeouts. And take a look at this

mtdeguzis commented 4 years ago

My use case is Icingaweb2 (Icinga2 web front end), the page loads, but the screenshot is taken so quick the alerts are not loaded: image

It would be nice to have the engine load the page, and allow the scrape request to wait X seconds.

Possibly related:

sbermind commented 4 years ago

Hello, try to play with --ajax-max-timeouts. And take a look at this

Hello! --ajax-max-timeouts works only for PhantomJS, isn't it?

I have the same problem. Some pages don't load because screenshots is done too fast. (i'm using -r chromium) It is good idea to add a request waiting. If you add this function i will be very grateful. Thank you.

zonicdoe commented 4 years ago

I'm working on a fork that will use the Devtools protocol to manage the screenshot process for Chromium based browsers such as Chrome. I've ran into the same issue in the past, specially with those sites that have some sort of WAF protection that is not very agressive, but it requires to wait for at least 5 seconds to render the requested page. With the aid of the Devtools protocol running the browser in remote debugging mode, some of the PhantomJS exclusive functions can be implemented in Chrome, such as image cropping, screenshot delay, http headers and cookies control, etc, etc. The only caveat I see with this implementation is the use of the "websockets" lib needed to interact with the browser, because it requires python 3, and it will break backwards compatibility with python 2 :( Do you guys know any websocket library compatible with python 2?

maaaaz commented 4 years ago

Python 2 compatibility does not have to maintained, particularly for a specific feature.

zonicdoe commented 4 years ago

Awesome :)

zonicdoe commented 4 years ago

Hello @maaaaz . Here is my suggested approach to implement the Devtools protocol in the script. It's not complete yet, and there are a lot of tests to run, but it will give you a general idea of where this is going: Running Chrome / Chromium in remote debugging mode: Line 325 Screenshot function (currently supporting PDF, JPG and PNG formats, image quality for JPG and screenshot delay): Line 765 I'll be implementing other functions from the PhantomJS renderer and then I will start the heavy testing, but for now I will be uploading new changes to the experimental branch. Regards ;) Edit: Added --crop option support: Line 950 Edit: Added --cookie option support: Line 828 Edit: Added --header option support: Line 815

MarcoLeder-zz commented 4 years ago

Dear maaaaz

Really looking forward for the implementation of this feature - hopefully you'll be able to add it soon :)