dotnetcore / DotnetSpider

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
MIT License
3.98k stars 1.04k forks source link

多个循环爬虫写在Quartz.net里 会内存溢出 #183

Closed tobytop closed 3 years ago

tobytop commented 4 years ago

如题

cjvega commented 4 years ago

本框架应该有自己的调度方式 如果需使用外层进行调度,根据实践,要修改下源码使用。可以控制内存增长。

Spider.cs

public abstract partial class Spider : BackgroundService
{
  ///
  public override void Dispose()
  {
    _requestedQueue.Dispose();
    base.Dispose();
    GC.Collect();
  }
}

Infrastructure/RequestedQueue.cs

public class RequestedQueue : IDisposable
{
  ///
  public void Dispose()
  {
    _timer.Stop();
    _timer.Dispose();
  }
}

同时希望获得作者采纳 @zlzforever

zlzforever commented 3 years ago

已采纳