search

dotnetcore / DotnetSpider

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
MIT License
3.98k stars 1.04k forks source link

添加SpiderOptions属性OneRequestDoneFirst，支持优先处理 request -> sub request ->… #244

Closed capadong closed 5 months ago

capadong commented 1 year ago

添加可选项，优先处理Depth深的请求，允许IDataFlow被提前处理，而不是放置到所有的请求之后。