zytedata / zyte-common-items

Contains the common item definitions used in Zyte.
BSD 3-Clause "New" or "Revised" License
9 stars 6 forks source link

Re-organize code #92

Closed kmike closed 6 months ago

kmike commented 6 months ago

This fixes GH-89.

Current approach:

  1. There is a Python module per item type in zyte_common_items.items. Each module contains item, sub-item (if applicable) and metadata class.
  2. Components are split into files. Related reusable components are put into modules, components specific to data types are put to separate files (e.g. components.real_estate).
  3. New converters module is added, with attrs converters; RequestCaster should be deprecated, and MetadataCaster is not used directly anymore
  4. request and metadata processors are moved to processors

TODO:

codecov-commenter commented 6 months ago

Codecov Report

Merging #92 (76d23b5) into main (8c37eed) will not change coverage. Report is 12 commits behind head on main. The diff coverage is 0.00%.

:exclamation: Current head 76d23b5 differs from pull request most recent head 5518d7d. Consider uploading reports for the commit 5518d7d to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #92 +/- ## ====================================== Coverage 0.00% 0.00% ====================================== Files 11 52 +41 Lines 1795 2055 +260 ====================================== - Misses 1795 2055 +260 ``` | [Files](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [zyte\_common\_items/\_\_init\_\_.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2F__init__.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvX19pbml0X18ucHk=) | `0.00% <ø> (ø)` | | | [zyte\_common\_items/components/brand.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Fbrand.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9icmFuZC5weQ==) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/pages/search\_request\_template.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fpages%2Fsearch_request_template.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvcGFnZXMvc2VhcmNoX3JlcXVlc3RfdGVtcGxhdGUucHk=) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/util.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Futil.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvdXRpbC5weQ==) | `0.00% <0.00%> (ø)` | | | [...yte\_common\_items/components/additional\_property.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Fadditional_property.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9hZGRpdGlvbmFsX3Byb3BlcnR5LnB5) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/components/gtin.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Fgtin.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9ndGluLnB5) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/processors.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fprocessors.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvcHJvY2Vzc29ycy5weQ==) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/components/breadcrumbs.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Fbreadcrumbs.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9icmVhZGNydW1icy5weQ==) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/components/real\_estate.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Freal_estate.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9yZWFsX2VzdGF0ZS5weQ==) | `0.00% <0.00%> (ø)` | | | [zyte\_common\_items/components/author.py](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree&filepath=zyte_common_items%2Fcomponents%2Fauthor.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-enl0ZV9jb21tb25faXRlbXMvY29tcG9uZW50cy9hdXRob3IucHk=) | `0.00% <0.00%> (ø)` | | | ... and [37 more](https://app.codecov.io/gh/zytedata/zyte-common-items/pull/92?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | |