search

vezaynk / Sitemap-Generator-Crawler

PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.
https://www.bbss.dev
MIT License
241 stars 92 forks source link

Option to ignore arguments #30

Closed vezaynk closed 7 years ago

vezaynk commented 7 years ago

Some websites do not require any GET arguments but use them for marketing purpose, a purpose not appropriate for sitemaps.

An option to ignore them should be introduced.