limber-mill and atlassian/localstack

[egut]

I'm trying to use lumber-mill with atlassian/localstack (AWS on localhost) but to do that I need to change endpoints for KCL (both kinesis and dynamoDB). Unfortunately gradle/java/groovy isn't my cop of tea so I thing I have narrow it down to this:

file: lumbermill-aws-kcl/src/main/java/lumbermill/aws/kcl/internal/KinesisConsumerBootstrap.java in start():

     AmazonKinesisClient kinesisClient = new AmazonKinesisClient(kinesisCfg.getKinesisCredentialsProvider(),
            kinesisCfg.getKinesisClientConfiguration());

    if(kinesisCfg.getKinesisEndpoint() != "") {
        kinesisClient.setEndpoint(kinesisCfg.getKinesisEndpoint());
    }

    AmazonDynamoDBClient dynamoDBClient = new AmazonDynamoDBClient(kinesisCfg.getDynamoDBCredentialsProvider(),
            kinesisCfg.getDynamoDBClientConfiguration());

    if(kinesisCfg.getDynamoDBEndpoint() != "") {
        dynamoDBClient.setEndpoint(kinesisCfg.getDynamoDBEndpoint());
    }

But from this point I'm a bit lost. Could you help me update the code?

[jrask]

Sorry for responding so late. The answer is quite simple, when you create KCL you can supply this to KinesisClientLibConfiguration.

Docs https://github.com/awslabs/amazon-kinesis-client/blob/master/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/KinesisClientLibConfiguration.java

Below is a complete sample with this info included.

// Uses minimal KCL configuration
KCL.create (
    new KinesisClientLibConfiguration (
        env ('appName', 'testApp').string(),
        env ('streamName', 'testStream').string(),
        new DefaultAWSCredentialsProviderChain(),
        workerId())
    .withRegionName(env ("region", "eu-west-1").string())

    // Custom endpoints
    .withDynamoDBEndpoint("custom")
    .withKinesisEndpoint("custom"))

    // Dry will not checkpoint
    .dry(env ("dry", "false").bool())

    // Each record as an observable
    .handleRecordBatch { record ->
        record

            // Print raw string
            .doOnNext{BytesEvent event -> println event.raw().utf8()}

            // Count and print count
            .count()
            .doOnNext{count -> println count}
    }

[jrask]

Closing this for now @egut