scrapinghub / spidermon

Scrapy Extension for monitoring spiders execution.
https://spidermon.readthedocs.io
BSD 3-Clause "New" or "Revised" License
526 stars 94 forks source link

Add Percentage Thresholds to `UnwantedHTTPCodesMonitor` #380

Closed mrwbarg closed 1 year ago

mrwbarg commented 1 year ago

Closes #376

Added option to set thresholds as a percentage of total spider requests' for the UnwantedHTTPCodesMonitor. The implementation follows the discussions on the issue and alos @curita's suggestions.

Added relevant unit tests.

codecov[bot] commented 1 year ago

Codecov Report

Base: 74.89% // Head: 74.98% // Increases project coverage by +0.08% :tada:

Coverage data is based on head (32d90b8) compared to base (062bb7d). Patch coverage: 100.00% of modified lines in pull request are covered.

:exclamation: Current head 32d90b8 differs from pull request most recent head bb03d31. Consider uploading reports for the commit bb03d31 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #380 +/- ## ========================================== + Coverage 74.89% 74.98% +0.08% ========================================== Files 73 73 Lines 3163 3174 +11 Branches 373 375 +2 ========================================== + Hits 2369 2380 +11 Misses 727 727 Partials 67 67 ``` | [Impacted Files](https://codecov.io/gh/scrapinghub/spidermon/pull/380?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub) | Coverage Δ | | |---|---|---| | [spidermon/contrib/scrapy/monitors.py](https://codecov.io/gh/scrapinghub/spidermon/pull/380?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub#diff-c3BpZGVybW9uL2NvbnRyaWIvc2NyYXB5L21vbml0b3JzLnB5) | `96.88% <100.00%> (+0.15%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

mrwbarg commented 1 year ago

It took me a bit to understand why each calculation is needed inside the monitor, maybe some comments explaining why would help

Done