search

awslabs / amazon-kinesis-client-ruby

A Ruby interface for the Amazon Kinesis Client Library. Allows developers to easily create robust application to process Amazon Kinesis streams in Ruby.
Apache License 2.0
146 stars 55 forks source link

Native Ruby KCL #27

Open henadzit opened 6 years ago

henadzit commented 6 years ago

Hello,

Are there plans on developing native Ruby Kinesis Client Library? Having the Java library running Ruby code when you have a simple Rails application is very far from optimal. It makes Docker instances heavy and requires lots of supporting code.

Thanks, Henadzi

EdgarOrtegaRamirez commented 6 years ago

Hey @henadzit, I'm not related with AWS in any way, but I have implemented several KCL apps with this repository. I don't think they will develop a pure Ruby version as this is not updated since almost a year ago.

Java is not running your Ruby code, it simply calls your ruby app using shell ruby script.rb, then they interchange messages using STDOUT.

My recommendation is that you should not integrate this kind of apps with Rails in any way, however you can connect to your Rails app database using the PG gem or Sequel.

henadzit commented 6 years ago

@EdgarOrtegaRamirez, thank you for your reply.

Java is not running your Ruby code, it simply calls your ruby app using shell ruby script.rb, then they interchange messages using STDOUT.

I get it but it doesn't change much. We still need to build heavy Docker instances and write lots of code. To be honest, that's quite a bummer, especially, since Kinesis isn't cheap at all. We're considering moving to Apache Kafka because it seems to have better client libraries.

EdgarOrtegaRamirez commented 6 years ago

Could you please describe heavy? You only need an Linux box with the Java SDK installed, nothing more. There is already a Rakefile setup you should use when creating your Dockerfile and if your app requires/has a lot of code that has nothing to do with this repo.

Apache Kafka on the other side is pretty great 👍, you should use it

marcotc commented 5 years ago

@EdgarOrtegaRamirez I'm assuming this is what @henadzit is referring to:

$ docker images
TAG              SIZE
before-kcl       519MB
after-kcl        947MB

Sitting at 428MB, amazon-kinesis-client-ruby is our largest dependency.

It seems like Amazon has a C++ implementation of the Kinesis client. I'm thinking that the C++ implementation could be used as a native extension to this gems, which would remove the requirement to run a JVM.

waka commented 4 years ago

Hi, I made kcl-rb that is a client library written by pure ruby.