search

everwatchsolutions / json-data-generator

A robust, generic, streaming random json data generator for your data
Apache License 2.0
397 stars 176 forks source link

Kinesis logger and config files #47

Closed data-tech closed 6 years ago

data-tech commented 6 years ago

Kinesis Logger with default credentials and role-based access. I have used AWS SDK for putting the records as this is the faster then KCL. For putting records as batches we can use KCL, for which we can add another logger with batching capability.

andrewserff commented 6 years ago

Hi @data-tech, thanks for the contribution. I appoligize I've been a bit behind on the PRs out there. I just merged the other two which has now caused merge conflicts with yours. Would you be willing to fix the conflict and push it again, then I'll merge this into master. Thanks again!

data-tech commented 6 years ago

Hi @andrewserff, I have resolved the conflicts and merged my code. Thanks.

andrewserff commented 6 years ago

Hi @data-tech I just noticed after merging this that you had a dependency for the entire aws-sdk. I changed it to this: 

<dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-kinesis</artifactId>
            <version>1.11.447</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-sts</artifactId>
            <version>1.11.447</version>
        </dependency>

It builds, but I don't have a way to test this out. If you have time, could you test it out with that as the dependency? I'm probably going to commit it this way, so when you get a change to test, just let me know if there are issues.