-
@pfifer I've been using Kinesis as my real time data streaming pipeline for over a year now, and I am consistently running into an issue where Kinesis shards seem to randomly stop processing data. I a…
-
When kinesis a kinesis shard is split (eg. when scaling) it marks the parent shard as CLOSED and client writes start going to the child shards.
Benthos does not appear to be reading from CLOSED shard…
-
We scaled up our Kinesis stream in us-east-1 from 340 shards to 435 shards yesterday. We kept our KCL service running throughout the resharding. After the previous open shards are closed, we found tha…
-
### Component(s)
exporter/awskinesis
### Is your feature request related to a problem? Please describe.
Currently spans of a single trace can be spread over multiple shards, so its not possible (or…
-
We have an implementation of dynamo db streams on a DynamoDB table. But often, especially on node restart, we have the exceptions like this and they do not stop:
`com.amazonaws.services.kinesis.cli…
-
We are in the process of upgrading several consumers to KCL 2.0. They are attached to a somewhat large stream (thousands of shards) and have been running on KCL 1.x for a long time without issue.
T…
echo8 updated
5 months ago
-
I have an access to AWS services via new user which my customer created for me. I don't know what permission I have. I created new stream and trying to start consumer.
Got next Exception
```
2019-…
-
Currently the S3 Loader buffers operate on a per-shard basis. This means that if your rotation buffer is 64mb each shard allocated to the consumer can consume this much memory. If the number of shar…
-
[HS 1474491803]
## Description
The following error is happening when running the AWS Kinesis/CloudWatch input on the Graylog Forwarder
> 2023-04-27T10:49:44.464-0500 INFO [aws-kinesis-reader-0] …
-
The main work recently (and monthly import) have involved running the bulk transformer, which transforms from the S3 files produced from buffering the Kinesis stream.
This means the app which consu…