search

my8100 / logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.
GNU General Public License v3.0
89 stars 22 forks source link

LogParser High CPU Usage #7

Closed aysark closed 5 years ago

aysark commented 5 years ago

Why does LogParser use up 100% of CPU all the time? This definitely shouldn't be needed. I have it running a vanilla build of scrapydweb with logparser and not running any jobs or anything and LogParser uses 100% all the time for days on...

my8100 commented 5 years ago

What’ the value of PARSE_ROUND_INTERVAL?

my8100 commented 5 years ago

Are there a lot of log files in SCRAPYD_LOGS_DIR? LogParser would parse the existing log files if they are not parsed yet.

my8100 commented 5 years ago

@aysark Could you share the logs of LogParser, as well as the screenshot of the CPU usage?