import attrs
from zyte_common_items import Product
from web_poet import handle_urls, WebPage, field
@handle_urls("example.com")
@attrs.define
class ProductPage(WebPage[Product]):
product: Product
@field
def name(self) -> str:
return f"(modified) {self.product.name}"
The common use case for this are fixing or modifying fields. However, if the item in question has a lot of fields, the Page Object code could be convoluted:
@handle_urls("example.com")
@attrs.define
class ProductPage(WebPage[Product]):
product: Product
@field
def name(self) -> str:
return f"(modified) {self.product.name}"
@field
def brand(self) -> str:
return self.product.brand
@field
def color(self) -> str:
return self.product.color
# and so on for the rest of the unmodified fields ...
We should provide/document an easy way to go about this. It'd be great if there are some built-in way to do this such that we don't have to re-iterate all of the unmodified fields with such boilerplate code.
An idea:
Decorator for the Page Object which auto-populates the item when to_item() is called. A bit tricky when individual fields are called. Also need to ensure the item validation mechanism is correct.
Following the https://github.com/scrapinghub/scrapy-poet/pull/164 PR being merged, we can now override items from providers (e.g. from scrapy-zyte-api's
ZyteApiProvider
):The common use case for this are fixing or modifying fields. However, if the item in question has a lot of fields, the Page Object code could be convoluted:
We should provide/document an easy way to go about this. It'd be great if there are some built-in way to do this such that we don't have to re-iterate all of the unmodified fields with such boilerplate code.
An idea:
to_item()
is called. A bit tricky when individual fields are called. Also need to ensure the item validation mechanism is correct.