search

my8100 / scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
https://github.com/my8100/files
GNU General Public License v3.0
3.15k stars 563 forks source link

[BUG] Too many jobs in the dashboard #134

Closed codehia closed 4 years ago

codehia commented 4 years ago

There are too many jobs in the dashboard all the completed ones as well, and all the jobs are sorted by the date in the order where the oldest jobs come first.

I use GIT version with scrapyd-deploy to deploy the spiders, and a custom command to run the crawl.

my8100 commented 4 years ago
  1. You can set up the below option: https://github.com/my8100/scrapydweb/blob/12c48923dd64bbef3c8db2bd40c93d1854ebd46f/scrapydweb/default_settings.py#L174-L177
  2. As the demo website shows, the latest finished jobs come first. http://scrapydweb.herokuapp.com/1/jobs/?style=classic