scrapinghub / web-poet

Web scraping Page Objects core library
https://web-poet.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
94 stars 15 forks source link

ZyteItemAdapter usage in fixtures #136

Open wRAR opened 1 year ago

wRAR commented 1 year ago

As found in #134, if ZyteItemAdapter is added to ItemAdapter when the fixture is generated, like documented at https://zyte-common-items.readthedocs.io/en/latest/setup.html#configuration , the generated output.json will not contain fields that are empty, and when running the test you need to also add ZyteItemAdapter to ItemAdapter, e.g. via conftest.py. At the same time, we may want to actually emit empty fields.

wRAR commented 1 year ago

Not sure what parts need to be changed for this after adding a ZyteTestsItemAdapter to zyte-common-items: as web-poet doesn't use zyte-common-items itself and doesn't depend on it (but zyte-common-items depends on web-poet), it doesn't seem right to use an adapter from it to the web-poet testing code. On the other hand, zyte-common-items doesn't configure ItemAdapter, only telling the user how to do that.