search

scrapinghub / spidermon

Scrapy Extension for monitoring spiders execution.
https://spidermon.readthedocs.io
BSD 3-Clause "New" or "Revised" License
533 stars 98 forks source link

no proper message or monitors for extra fields when using jsonschema #227

Open HassanQamar07 opened 5 years ago

HassanQamar07 commented 5 years ago

The stats for unexpected fields is almost unreadable and can't be used for writing a monitor to raise error if a crawler is crawling any unexpected field.

'spidermon/validation/fields/errors/unexpected_fields_upopularity_ranking_uratings_utotal_competition_ucertificate_of_excellence_unum_of_photos_uclaimed_uprice_range_were_unexpected': 1, 'spidermon/validation/fields/errors/unexpected_fields_upopularity_ranking_uratings_utotal_competition_ucertificate_of_excellence_unum_of_photos_uclaimed_uprice_range_were_unexpected/': 1,

rennerocha commented 4 years ago

Do you have some suggestion a better way to include these error in the stats?