Support robots.txt - Githubissues

davidglezz / tfg

Prototype of a web application that indexes microdata (schema.org) exposed by electronic stores and allows the search of products.

MIT License

3 stars 2 forks source link

Open davidglezz opened 6 years ago

davidglezz commented 6 years ago

Support robots.txt to get the sitemap url and restrictions.

davidglezz commented 6 years ago