scrapinghub / spidermon

Scrapy Extension for monitoring spiders execution.
https://spidermon.readthedocs.io
BSD 3-Clause "New" or "Revised" License
526 stars 94 forks source link

Adding use of ItemAdapter to prevent assumptions of item nature #358

Closed further-reading closed 1 year ago

further-reading commented 1 year ago

https://github.com/scrapinghub/spidermon/issues/353

codecov[bot] commented 1 year ago

Codecov Report

Base: 74.28% // Head: 74.62% // Increases project coverage by +0.34% :tada:

Coverage data is based on head (454eefc) compared to base (5020fca). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #358 +/- ## ========================================== + Coverage 74.28% 74.62% +0.34% ========================================== Files 73 73 Lines 3126 3113 -13 Branches 487 366 -121 ========================================== + Hits 2322 2323 +1 + Misses 737 725 -12 + Partials 67 65 -2 ``` | [Impacted Files](https://codecov.io/gh/scrapinghub/spidermon/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub) | Coverage Δ | | |---|---|---| | [spidermon/contrib/scrapy/extensions.py](https://codecov.io/gh/scrapinghub/spidermon/pull/358/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub#diff-c3BpZGVybW9uL2NvbnRyaWIvc2NyYXB5L2V4dGVuc2lvbnMucHk=) | `85.41% <100.00%> (+0.15%)` | :arrow_up: | | [spidermon/contrib/scrapy/pipelines.py](https://codecov.io/gh/scrapinghub/spidermon/pull/358/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub#diff-c3BpZGVybW9uL2NvbnRyaWIvc2NyYXB5L3BpcGVsaW5lcy5weQ==) | `83.50% <100.00%> (+7.14%)` | :arrow_up: | | [spidermon/contrib/validation/jsonschema/formats.py](https://codecov.io/gh/scrapinghub/spidermon/pull/358/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub#diff-c3BpZGVybW9uL2NvbnRyaWIvdmFsaWRhdGlvbi9qc29uc2NoZW1hL2Zvcm1hdHMucHk=) | `100.00% <0.00%> (+28.57%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scrapinghub)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

kmike commented 1 year ago

hey! Would it be possible to add tests for it?

further-reading commented 1 year ago

It's still a work in progress haha, of course there'll be tests. We can hold off on the reviews until its ready.

VMRuiz commented 1 year ago

Looks good to me.

I would however add itemadapter as a dependency now. Things will work either way because of the dependency with Scrapy, but it makes things more future-proof (e.g. in the unlikely case that Scrapy drops itemadapter as a dependency).

@further-reading Could you please fix the missing dependency in setup.py ?

Gallaecio commented 1 year ago

@further-reading Unless you plan on further changes, e.g. https://github.com/scrapinghub/spidermon/pull/358#discussion_r968693785, we can merge as far as I am concerned.

further-reading commented 1 year ago

@Gallaecio apologies I missed that one. Should be sorted out now

Gallaecio commented 1 year ago

Thank you!