search

acryldata / datahub-actions

DataHub Actions is a framework for responding to changes to your DataHub Metadata Graph in real time.
42 stars 47 forks source link

feat(action): add metadata_change_sync action to sync two datahub instances #72

Closed TonyOuyangGit closed 1 year ago

TonyOuyangGit commented 1 year ago

This PR implements a new action that can listen to the MetadataChangeLogEvent_v1 events and convert it to MetadataChangeProposal and use the Python emitter SDK to emit the proposal to another datahub instance.

The configuration file takes gms_server and gms_auth_token to emit the proposal to another datahub instance, since MetadataChangeProposal currently only supports the UPSERT change type, we'll only focus on MetadataChangeLogEvent_v1 with the UPSERT type.

Tests and doc for the changes have been added

github-actions[bot] commented 1 year ago

Unit Test Results (build & test)

63 tests  +2   63 :heavy_check_mark: +2   3s :stopwatch: ±0s   1 suites ±0     0 :zzz: ±0    1 files   ±0     0 :x: ±0 

Results for commit 7117553b. ± Comparison against base commit 4490e9ba.

:recycle: This comment has been updated with latest results.

jinlintt commented 1 year ago

Thanks @hsheth2 and team for reviewing this! We are excited to seeing this merged.