search

cldellow / datasette-scraper

Add website scraping abilities to Datasette
Apache License 2.0
60 stars 1 forks source link

plugin: discover-only-same-origin #15

Closed cldellow closed 1 year ago

cldellow commented 1 year ago 
      // When true, only follows links to pages on the same domain
      // as the source page
      "discover-only-same-origin": true

Needs https://github.com/cldellow/datasette-scraper#canonicalize_urlconfig-from_url-to_url-to_url_depth