search

paceaux / selector-finder

Find a CSS selector on a public site
MIT License
23 stars 3 forks source link

Use an existing crawl results if available #6

Closed paceaux closed 2 years ago

paceaux commented 2 years ago

When crawling the site, first check to see if the results of crawling are already present. If they are present, load that file, instead of recrawling.

optionally give an option to use a file?

paceaux commented 2 years ago

now it checks <site>.sitemap.json first, before generating it. User has the option to override.

this effectively gives user to also provide own list of links.