search

awslabs / emr-dynamodb-connector

Implementations of open source Apache Hadoop/Hive interfaces which allow for ingesting data from Amazon DynamoDB
Apache License 2.0
216 stars 135 forks source link

Add support for custom configuration of the underlying DynamoDbClientBuilder #196

Closed julienrf closed 1 month ago

julienrf commented 4 months ago

Introduced a new configuration key containing the class name of a transformer to apply to the underlying DynamoDbClientBuilder used by our DynamoDBClient class.

Description of changes:

Currently, it is possible to customize the underlying DynamoDbClient used by the connector by using some settings in the job configuration (e.g. DynamoDBConstants.ENDPOINT, or even DynamoDBConstants.CUSTOM_CREDENTIALS_PROVIDER_CONF).

However, some customization are still impossible to achieve in the current situation. For instance, it is not possible to set a custom EndpointProvider, or to enable endpoint discovery, or to set a custom auth scheme provider…

This PR addresses these limitations by introducing a new hook allowing the users to have full control over the DynamoDbClientBuilder before the connector uses it to build the actual DynamoDbClient.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

julienrf commented 3 months ago

May I get a review from the maintainers, please?

/cc @chloeyamzn @srbgupta86