dotnetcore / DotnetSpider

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
MIT License
3.98k stars 1.04k forks source link

获取指定域名全站的url #203

Closed guxing2 closed 3 years ago

guxing2 commented 3 years ago

可以只输入域名来获取当前站点的所有的url吗?

zlzforever commented 3 years ago

你可以了解一下:网站遍历的逻辑、算法,有深度优先和广度优先。