search

The-Academic-Observatory / oaebu-workflows

Telescopes, Workflows and Data Services for the 'Book Analytics Dashboard Project (2022-2025)', building upon the project 'Developing a Pilot Data Trust for Open Access eBook Usage (2020-2022)'
https://documentation.book-analytics.org/
Apache License 2.0
5 stars 0 forks source link

NOT NULL statements for crossref and GA #202

Closed keegansmith21 closed 11 months ago

keegansmith21 commented 11 months ago

JIRA ticket

@kathrynnapier noticed that some entries in the export tables were missing. This appears to be because they're missing an IF NOT NULL statement for crossref and GA.

codecov[bot] commented 11 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (0b1ef28) 93.18% compared to head (727e53e) 93.18%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #202 +/- ## ======================================= Coverage 93.18% 93.18% ======================================= Files 16 16 Lines 2743 2743 Branches 390 390 ======================================= Hits 2556 2556 Misses 103 103 Partials 84 84 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

kathrynnapier commented 11 months ago

@keegansmith21 - I think I may have missed jstor as well, for the author export table anyway. Comparing with the export_book_metrics.sql.jinja2, should array_length be used instead?

OR ARRAY_LENGTH(month.crossref_events) > 0 OR ARRAY_LENGTH(month.jstor_country) > 0

keegansmith21 commented 11 months ago

@keegansmith21 - I think I may have missed jstor as well, for the author export table anyway. Comparing with the export_book_metrics.sql.jinja2, should array_length be used instead?

OR ARRAY_LENGTH(month.crossref_events) > 0 OR ARRAY_LENGTH(month.jstor_country) > 0

@kathrynnapier - Yep jstor condition was also missing for the export tables. I have also changed the conditions to check the array length for crossef and jstor rather than check if they are null.