set header info for fetch()

eklem / browsercrawler

Crawling content from a site within the browser. A basis for i.e. a search solution for static sites.

https://eklem.github.io/browsercrawler/doc/

MIT License

2 stars 0 forks source link

Open eklem opened 6 years ago

eklem commented 6 years ago

So it can be identified by server that is crawled.

eklem commented 6 years ago

crawler-in-browser site crawl, v0.0.x

The function needs to take an email address (mandatory) for site owner so server owner can get in contact.