Closed mumoshu closed 6 years ago
This PR depends on #28
Merging #31 into master will increase coverage by
29.73%
. The diff coverage is73.68%
.
@@ Coverage Diff @@
## master #31 +/- ##
===========================================
+ Coverage 33.89% 63.63% +29.73%
===========================================
Files 4 4
Lines 118 132 +14
===========================================
+ Hits 40 84 +44
+ Misses 71 42 -29
+ Partials 7 6 -1
Impacted Files | Coverage Δ | |
---|---|---|
streams/client.go | 68.81% <73.68%> (+43.5%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 028b7c1...a4967d5. Read the comment docs.
This addresses a few issues I had encountered in my production testing. As I'm satisfied with the implementation and tests are added for additional coverage, I'll be happily merging this tomorrow :)
@s12v Thanks for reviewing 👍
Turns out awsbeats was calling
batch.RetryEvents
on different data and timing. Revise the implementation according to the official beat outputsFixes #29
I've also added a new-line per each kinesis record data.
Firehose doesn't automatically add trailing new-line on after each record. This ends up a stream->firehose->s3 pipeline to produce useless s3 objects. Not ndjson, but a sequence of json objects without separators... Fix it just adding a new-line.