DannyBen / snapcrawl

Crawl a website and take screenshots
MIT License
57 stars 12 forks source link

Add Ability to open local files #45

Closed jakegrabowski closed 2 years ago

jakegrabowski commented 2 years ago

If is not much work it would be nice to have possibility opening files directy from hdd.

Current behavior

docker run --rm -it --volume "$PWD:/app" dannyben/snapcrawl file:///Users/jake/Dropbox/Docker/snapcrawl/index.html
 INFO : processing http://file:///Users/jake/Dropbox/Docker/snapcrawl/index.html, depth: 0
ERROR : http error on http://file:///Users/jake/Dropbox/Docker/snapcrawl/index.html - SocketError: Failed to open TCP connection to file:80 (getaddrinfo: Name or service not known)
DannyBen commented 2 years ago

Yes, perhaps. I might take a look at implementing it.

An alternative would be to run a simple lightweight static web server. Since I see you are using docker already, its just a matter of running another container.

One such server is httpme which can also be executed via docker:

$ docker run --rm -it -p 3000:3000 -v $PWD:/docroot dannyben/httpme

And then you snapcrawl on http://localhost:3000

jakegrabowski commented 2 years ago

Thanks for posting example solution for docker. Please note that users without docker still are unable to crawl local files.

DannyBen commented 2 years ago

Local file screenshots was never a priority, and people without docker can just run a local server without docker (same one even, if they are on Linux or Mac).

$ gem install httpme
$ httpme

In fact, using it like this was the original intention of snapcrawl - to be able to create screenshots of a local development server.

jakegrabowski commented 2 years ago

Yeah I know. It's not priority for me either. Just took a time to make note about this use case because you took time to publish your work.

DannyBen commented 2 years ago

I have looked at the code, adding support for local HTML is beyond the scope of snapcrawl - will require too many changes. I am closing this, but thanks for bringing it to my attention - I might revisit it in the future.