The-Academic-Observatory / oaebu-workflows

Telescopes, Workflows and Data Services for the 'Book Analytics Dashboard Project (2022-2025)', building upon the project 'Developing a Pilot Data Trust for Open Access eBook Usage (2020-2022)'
https://documentation.book-analytics.org/
Apache License 2.0
5 stars 0 forks source link

Add collections to JSTOR Telescope #197

Closed keegansmith21 closed 11 months ago

keegansmith21 commented 1 year ago

JSTOR has added support for collections of works, as opposed to their normal functioning regime which involves grouping works by publisher.

From a technical perspective, Collection reports are a little different to publisher reports, but they are not different enough (and the metrics they provide are 1:1 comparable) to warrant a new telescope. Instead, I implemented an option for the current JSTOR telecope that makes room for processing a collection. The collection and publisher-unique functionality is contained by their own classes, which each inherit from the JstorAPI class.

Implementation Notes

Data Partners

In the config, the following kwargs should be changed for JSTOR DAGs:

- publisher_id
+ entity_id: 
+ entity_type:
+ country_partner: 
+ institution_partner:

Where:

The jstor_country_collection and jstor_institution_collection data partners were added for the sole purpose of directing the telescope to the correct schema for table upload. The schemas are slightly different, so this was necessary. It is important to note, however, that the ONIX WF should use the original (jstor_country, jstor_institution) data partners regardless. This is because the two are treated as the same in the ONIX WF - the difference in fields is not important as it only processes ISBN usage.

Regex:

The regex that matches on report name is provisional. I only have one example email to base this off so it may change as needed when onboarding future collections.

codecov[bot] commented 11 months ago

Codecov Report

Attention: 25 lines in your changes are missing coverage. Please review.

Comparison is base (f9d26d8) 93.29% compared to head (a033fea) 94.27%.

:exclamation: Current head a033fea differs from pull request most recent head 44a7263. Consider uploading reports for the commit 44a7263 to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #197 +/- ## ========================================== + Coverage 93.29% 94.27% +0.97% ========================================== Files 16 16 Lines 2641 2480 -161 Branches 376 325 -51 ========================================== - Hits 2464 2338 -126 + Misses 95 79 -16 + Partials 82 63 -19 ``` | [Files](https://app.codecov.io/gh/The-Academic-Observatory/oaebu-workflows/pull/197?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The-Academic-Observatory) | Coverage Δ | | |---|---|---| | [oaebu\_workflows/oaebu\_partners.py](https://app.codecov.io/gh/The-Academic-Observatory/oaebu-workflows/pull/197?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The-Academic-Observatory#diff-b2FlYnVfd29ya2Zsb3dzL29hZWJ1X3BhcnRuZXJzLnB5) | `96.15% <ø> (ø)` | | | [oaebu\_workflows/workflows/jstor\_telescope.py](https://app.codecov.io/gh/The-Academic-Observatory/oaebu-workflows/pull/197?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The-Academic-Observatory#diff-b2FlYnVfd29ya2Zsb3dzL3dvcmtmbG93cy9qc3Rvcl90ZWxlc2NvcGUucHk=) | `93.02% <90.31%> (-1.07%)` | :arrow_down: | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/The-Academic-Observatory/oaebu-workflows/pull/197/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The-Academic-Observatory)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.