Setting up a class for scraped items instead of using a dict can make it easier for contributors to know what needs to be pulled from a given page. It looks like scrapy also supports using Python dataclasses for items, which lets us set up type validation without any extra work as long as we're on Python 3.7 or above (or using a backport for Python 3.6).
I can give this a try this week if that works. Let me know!
easherma
commented
4 years ago
Setting up a class for scraped items instead of using a
dictcan make it easier for contributors to know what needs to be pulled from a given page. It looks like scrapy also supports using Python dataclasses for items, which lets us set up type validation without any extra work as long as we're on Python 3.7 or above (or using a backport for Python 3.6).I can give this a try this week if that works. Let me know!