search

acryldata / datahub-actions

DataHub Actions is a framework for responding to changes to your DataHub Metadata Graph in real time.
42 stars 47 forks source link

feat: add configuration for Kafka consumer groups via environment variables #112

Closed Masterchen09 closed 5 months ago

Masterchen09 commented 8 months ago

To have configurable consumer groups, this PR introduces three new environment variables which are used for the names of the three existing pipelines - the names of the pipelines are used for the consumer groups:

DATAHUB_ACTIONS_INGESTION_EXECUTOR_CONSUMER_GROUP_ID (ingestion_executor) DATAHUB_ACTIONS_SLACK_CONSUMER_GROUP_ID (datahub_slack_action) DATAHUB_ACTIONS_TEAMS_CONSUMER_GROUP_ID (datahub_teams_action)

If the environment variables are not provided the existing pipeline names will be used by default.

There will be another PR in the datahub-helm repository which will use these environment variables.

edit: The other PR in the datahub-helm repository.

github-actions[bot] commented 8 months ago

Unit Test Results (build & test)

63 tests  ±0   63 :heavy_check_mark: ±0   3s :stopwatch: ±0s   1 suites ±0     0 :zzz: ±0    1 files   ±0     0 :x: ±0 

Results for commit ec9596ba. ± Comparison against base commit 18c118d3.

:recycle: This comment has been updated with latest results.