if the CSS selector is used and the container is taller than the viewport, it seems to cut it

simonw / shot-scraper

A command-line utility for taking automated screenshots of websites

https://shot-scraper.datasette.io

Apache License 2.0

1.68k stars 75 forks source link

if the CSS selector is used and the container is taller than the viewport, it seems to cut it #19

Open simonw opened 2 years ago

simonw commented 2 years ago

Tried this, works great! One issue I found is that if the CSS selector is used and the container is taller than the viewport, it seems to cut it.

https://twitter.com/journeyer/status/1501772311715717126

Lewiscowles1986 commented 2 years ago

I saw you post something on twitter with an argument, that seemed to be a viewbox. Was that to work around this?

This tweet about release 0.6 shows a --padding option
The next tweet shows a --width property

Assuming the user cannot pad then width their way out of this, could a height be taken that:

if viewport was < height, then the difference would be subtracted (I guess for elements where CSS offset differs from where an anchor might take the user.
if viewport is < height, then starting at rect, and continuing until height reached.

Alternatively, you've shown the ability to run JS against a page. Can $0.getBoundingClientRect() (with $0 being the element specified) get the height, so you can arbitrarily enlarge the viewport to that size?

simonw commented 2 years ago

The challenge here is that the viewport size is set before the JavaScript starts executing: https://github.com/simonw/shot-scraper/blob/e40070d9bc5334ad8b2ff9e4e3565263aad545f4/shot_scraper/cli.py#L464-L476

It looks to me like the mechanism that takes a screenshot of an element based on its selector can't take screenshots that are larger than the current browser viewport:

https://github.com/simonw/shot-scraper/blob/e40070d9bc5334ad8b2ff9e4e3565263aad545f4/shot_scraper/cli.py#L496-L499

But I think the viewport size can be modified AFTER the page has run JavaScript. So it might be possible to inject the element that we are going to take the shot of, then evaluate its size as you suggest, then adjust the viewport, and then go ahead with taking the screenshot.

Lewiscowles1986 commented 2 years ago

thank you,

How would you feel if I were to work on a PR, where:

additional flag for this behavior (sadly, I think it's the sort of thing people will need to know up-front for now)
flag causes sub-process to be launched to get the element size
flag then launches separate sub-process to get the screenshot with element

I Think if you have an example Url + cli example of this, it would be something to help me work on it. I Can manage the sub-processing parts in python.