Closed keegansmith21 closed 1 year ago
Patch coverage: 100.00
% and project coverage change: +0.10
:tada:
Comparison is base (
46cd0a8
) 94.24% compared to head (37995b6
) 94.34%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
The fix from #144 has unfortunately failed in production as BigQuery gives an error when inserting too many rows into a temporary table (query too complex). This PR takes a new approach (and what I believe to be a cleaner one) to the issue. Instead of retrieving the ISBNs from the ONIX feed in python, this is collapsed into the SQL query. This removed the necessity of the isbns_from_onix() function entirely (it's not used for anything else). Furthermore, @jdddog mentioned that it'd be better to simply make the metadata table directly from the query (using create_bigquery_table_from_query()). I have implemented this change as well, which simplifies the metadata table creation significantly. The metadata transform functions are no longer necessary as the data should be transformed prior to the master metadata table creation.