Doesn't work on Scrapy Cloud with recent stacks

scrapy-plugins / scrapy-jsonschema

Scrapy schema validation pipeline and Item builder using JSON Schema

BSD 3-Clause "New" or "Revised" License

44 stars 12 forks source link

Doesn't work on Scrapy Cloud with recent stacks #3

Closed wRAR closed 5 years ago

wRAR commented 7 years ago

JsonSchemaItem is a subclass of scrapy.item.DictItem while a recent enough HubstorageExtension checks whether an item is a scrapy.Item (which is a subclass of DictItem too): https://github.com/scrapinghub/scrapinghub-entrypoint-scrapy/commit/52e5362c9872047efb2c45fe066a50c61bf41d49

redapple commented 7 years ago

Interesting @wRAR . I'm actually not sure where it should be fixed (if not in multiple places):

make JsonSchemaItem a subclass of scapy.item.Item
HubstorageExtension accept scapy.item.BaseItem

wRAR commented 7 years ago

@vshlapakov you may be interested in this

vshlapakov commented 7 years ago

If I recall it correctly, the code was added mainly to prevent wrong handling None items, so I'm inclined to update HubstorageExtension to check for BaseItem instead of Item, it looks consistent - I think it's fine to subclass custom items from DictItem, but I can miss something. /cc @chekunkov

chekunkov commented 7 years ago

@vshlapakov agree with you, let's change the HubstorageExtension