search

kimmobrunfeldt / url-to-image

PhantomJS screenshotting done right
https://www.npmjs.com/package/url-to-image
MIT License
204 stars 42 forks source link

url-to-image bails out partway through long page #13

Open holtzermann17 opened 8 years ago

holtzermann17 commented 8 years ago

$ urltoimage https://terrytao.wordpress.com/2013/11/19/polymath8b-bounded-intervals-with-many-primes-after-maynard/ maynard.png

This produces a very long, thin PNG image - approximately as desired - but it ends before the page is complete. In particular, it stops partway through this comment, which is about ⅔ of the way through the page.

screenshot 2016-11-03 09 53 09

Please note that this page is long (43pp if I press "Print" in Chrome), and also full of mathematical notation in case that matters. I'm assuming that urltoimage runs out of memory somewhere along the line.

Is there a way to get it to e.g. complete several PNG images that can be stitched together later, rather than quitting?

(Note, I've tried printing this page with various other extensions and online services and they all failed to generate any output whatsoever, so urltoimage has a leg up on the competition at the moment.)

joeytwiddle commented 8 years ago

It bails out because the default timeout is reached. Increasing the timeout to 5 minutes worked for me:

urltoimage --kill-timeout 300000 ...
holtzermann17 commented 8 years ago

It bails out because the default timeout is reached. Just increasing the timeout worked for me:

urltoimage --kill-timeout 600000 ...

Here I still see the truncated output with that modification.

joeytwiddle commented 8 years ago

Oh you are right. On your page, it looks like it's stopping at 32768 which is a limit I've heard of before with webdriver screenshots (2^15).

I have been playing with url2img today, which is a fork of this package. It does indeed support cropping, and this got the second half of the page for me:

url2img --cropoffset-top 35000 --kill-timeout 600000 https://terrytao.wordpress.com/2013/11/19/polymath8b-bounded-intervals-with-many-primes-after-maynard/ maynard2.png
kimmobrunfeldt commented 8 years ago

Thanks for reporting. I tried to search if this height limit could be increased but couldn't find anything. Those crop parameters were implemented in a PR for url-to-image too but somehow I can't get them working now. It would be nice to have the url2img merged back to this repo as it is a direct fork from this.