search

TeamHG-Memex / scrapy-rotating-proxies

use multiple proxies with Scrapy
MIT License
738 stars 158 forks source link

Log proxy stats only if ROTATING_PROXY_LOGSTATS_INTERVAL is non-zero #41

Closed StasDeep closed 4 years ago

StasDeep commented 4 years ago

When inside scrapy shell standard logs are disabled by setting LOGSTATS_INTERVAL to 0. When the same logic is used for ROTATING_PROXY_LOGSTATS_INTERVAL, the console is flooded with endless logs.

The only workaround for now, is to use a very big value for this setting, when in shell. However, I think it would be more natural to follow an example from Scrapy and use 0 as an indicator to disable logging.

codecov-commenter commented 4 years ago

Codecov Report

Merging #41 into master will decrease coverage by 0.85%. The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master      #41      +/-   ##
==========================================
- Coverage   51.68%   50.82%   -0.86%     
==========================================
  Files           5        5              
  Lines         238      242       +4     
  Branches       39       41       +2     
==========================================
  Hits          123      123              
- Misses        109      113       +4     
  Partials        6        6
Impacted Files Coverage Δ
rotating_proxies/middlewares.py 24.24% <0.00%> (-0.76%) :arrow_down:
StasDeep commented 4 years ago

@kmike could you please review it?

kmike commented 4 years ago

Looks great, thanks @StasDeep!