mhart / kinesalite

An implementation of Amazon's Kinesis built on LevelDB
MIT License
808 stars 86 forks source link

Is any memory limitation on putting data into kinesis stream with put-record? #57

Closed lucianonet closed 6 years ago

lucianonet commented 6 years ago

I tried to put content of a file that has more than 260 kb but I got an error. The put-record commands looks like: aws kinesis put-record --stream-name kinesis_stream_name --data "$(cat file.xml)" --partition-key "test" --endpoint-url http://127.0.0.1:4567 --no-verify-ssl --profile local And the error is: -bash: /usr/local/bin/aws: Argument list too long. For files that are smaller that 260 kb seems to work fine. I removed from file.xml some content and after that worked properly.

mhart commented 6 years ago

That error looks like it's coming from the AWS CLI tool to me?

lucianonet commented 6 years ago

Yes, but I thought that is related to kinesalite which is used behind the scene.

mhart commented 6 years ago

No, that error's saying the argument list that you're passing to the aws tool is too long - it hasn't even reached kinesalite yet

lucianonet commented 6 years ago

Thank you.