search

PicnicSupermarket / diepvries

The Picnic Data Vault framework.
https://diepvries.picnic.tech
MIT License
126 stars 15 forks source link

Merge similar DML queries #15

Closed dlouseiro closed 3 years ago

dlouseiro commented 3 years ago

Context

The current implementation of diepvries uses a different DML template for hub and link queries. Although, these queries are really similar so the multiple templates can be unified into a single template.

Solution

The following changes were performed:

  1. Unify src/diepvries/template_sql/hub_dml.sql, src/diepvries/template_sql/link_dml.sql and src/diepvries/template_sql/role_playing_hub_dml.sql into a single template (src/diepvries/template_sql/hub_link_dml.sql);
  2. Adapt hub.py, link.py and role_playing_hub.py to calculate the SQL placeholders for the new template SQL;
  3. Adjust test suite for new templates;
  4. Rename data_vault_table to target_table in all SQL templates. This change was not strictly needed, but makes sense to keep the placeholder for the schema and table aligned (target_schema/target_table instead of target_schema/data_vault_table).
  5. (while there) Rename role_playing_hub_load_sql to test_role_playing_hub_load_sql. This test function was not prefixed with test, hence was not being picked up by pytest.
  6. (while there) Merge data_vault_load tests with and without role playing hub. There was a lot of unnecessary redundancy between these two tests.

Dependencies

Depends on https://github.com/PicnicSupermarket/diepvries/pull/14