search

ScottMansfield / widow

Distributed, asynchronous web crawler
GNU Lesser General Public License v2.1
26 stars 4 forks source link

Support for sitemap.xml #4

Open ScottMansfield opened 9 years ago

ScottMansfield commented 9 years ago

https://support.google.com/webmasters/answer/156184?hl=en

ScottMansfield commented 9 years ago

This is being done in the exo project

ScottMansfield commented 9 years ago

work done in

22b72fd9f0a43540a39896a8f9e3978a2358503c 5af862347c29e5d215264b90f61c4bcedb97a62e