18F / rdbms-subsetter

Generates a subset of a relational database that respects foreign key constraints
Creative Commons Zero v1.0 Universal
313 stars 30 forks source link

Add event handler and trigger event row_will_be_added #30

Closed brki closed 8 years ago

brki commented 8 years ago

This addresses #29 .

To give you an idea of how I'm using this, here is the event handler file I'm using:

import sqlalchemy as sa

def row_added(source_db, source_row, target_db, target_table, prioritized):
    # Nothing to do unless this is a versioned table.
    if 'version_start_date' not in source_row:
        return

    # Add all previous / future versions.
    has_previous_versions = source_row.version_start_date != source_row.version_birth_date
    has_future_versions = source_row.version_end_date is not None and source_row.id != source_row.identity
    if has_previous_versions or has_future_versions:
        versions_select = sa.sql.select([target_table]).where(sa.and_(target_table.c.identity == source_row.identity, target_table.c.id != source_row.id))
        for version_row in source_db.conn.execute(versions_select):
            source_db.create_row_in(version_row, target_db, target_table)
brki commented 8 years ago

Note that merging this and #28 will cause a merge conflict. A correctly merged version of subsetter.py can be found here: https://github.com/brki/rdbms-subsetter/blob/event-handler-and-write-buffer-disabling/subsetter.py

jmcarp commented 8 years ago

This seems like a useful feature. What do you think about implementing using an existing library like blinker (https://github.com/jek/blinker), instead of custom code? Then, instead of a specific event_module argument, we can use something more generic like --import, so that modules of signal receivers are imported.

brki commented 8 years ago

@jmcarp that sounds like a good idea, I'll take a look at blinker.

catherinedevlin commented 8 years ago

Thanks for more great work! And thanks @jmcarp for the valuable advice!

catherinedevlin commented 8 years ago

Replaced by https://github.com/18F/rdbms-subsetter/pull/33 (which was merged), so closing.