c4software / python-sitemap

Mini website crawler to make sitemap from a website.
GNU General Public License v3.0
362 stars 110 forks source link

Images from different domains should not be added to sitemap #41

Closed ghost closed 7 years ago

ghost commented 7 years ago

Sitemap should contain only URLs that belong to the same domain and are under the current directory where the sitemap is located. The same rule applies to images and videos. Currently the script adds all images not checking the domain or directory.

See https://www.sitemaps.org/protocol.html