ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

[Sandeepgsn]

While executing below command: jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem

Getting error: ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

[marcosnils]

@Sandeepgsn try using the GEM in rubygems https://rubygems.org/gems/logstash-input-dynamodb

[sgarg-atlassian]

@marcosnils I changed the version in the gemspec file from s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.0.0' to s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.3.4' and it worked. Do you see any problems with this?

[marcosnils]

@sgarg-symphy I'm not the owner of this repo, I just maintain forked versions of this library because seems like AWS dropped support for this. I've pushed the gems in ruby gems and the ones there work for me.

[vamcyimmanni]

Struggling with the same issue. Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository. How to get it to work?

@marcosnils I have tried using the gems from https://rubygems.org/gems/logstash-input-dynamodb. That didn't work either. If possible can you please list out the steps.

Thank you

[marcosnils]

@vamcyleonardo how are you installing the gem? and in which logstash version?

[marcosnils]

@vamcyleonardo just updated the logstash-input-dynamodb gem in rubygems. Try now.

[vamcyimmanni]

Thank you for your response. I will try it.

[marcosnils]

@vamcyleonardo don't clone that repo, install the gem from rubygems. Which logstash version are you using?

[marcosnils]

@vamcyleonardo use the gem from ruby gems. Don't clone and build from this repo.

[marcosnils]

gem install logstash-input-dynamodb:'> 2' plugin install logstash-input-dynamodb

[marcosnils]

gem is there https://rubygems.org/gems/logstash-input-dynamodb. Try removing the :'> 2' from the end

[marcosnils]

maybe, give it a try

[marcosnils]

@vamcyleonardo it should work, that's how I build my logstash docker images. Maybe if you try a clean installation it works.

If you don't want to deal with all this :hankey: you can just use our docker image https://hub.docker.com/r/mantika/logstash-dynamodb-streams/

[marcosnils]

U sure the credentials you're specifying have the correct access?

These are the policies you need

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "pointerTable0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGetItem",
                "dynamodb:BatchWriteItem",
                "dynamodb:DeleteItem",
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:CreateTable",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": [
                "<your arn>:table/*cptr"
            ]
        },{
            "Sid": "streamReading1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:Scan",
                "dynamodb:ListStreams",
                "dynamodb:GetShardIterator",
                "dynamodb:GetRecords"
            ],
            "Resource": [
                "<table ARN>",
                "<stream ARN>"
            ]
        }
    ]
}

[marcosnils]

Remember that you also need to enable the table stream in dynamoDB

[mcolarusso]

When running the docker image, i get "Couldn't find any input plugin named 'dynamodb'". Any idea why?

[marcosnils]

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" notifications@github.com wrote:

When running the docker image, i get "Couldn't find any input plugin named 'dynamodb'". Any idea why?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/logstash-input-dynamodb/issues/21#issuecomment-242227040, or mute the thread https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2 .

[mcolarusso]

I tried latest and 2

From: Marcos Nils Reply-To: awslabs/logstash-input-dynamodb Date: Wednesday, August 24, 2016 at 6:56 PM To: awslabs/logstash-input-dynamodb Cc: Michael Colarusso, Comment Subject: Re: [awslabs/logstash-input-dynamodb] ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository (#21)

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" notifications@github.com<mailto:notifications@github.com> wrote:

When running the docker image, i get "Couldn't find any input plugin named 'dynamodb'". Any idea why?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/logstash-input-dynamodb/issues/21#issuecomment-242227040, or mute the thread https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/awslabs/logstash-input-dynamodb/issues/21#issuecomment-242233947, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFVq4jXMtza36zX3kN9XvYtQD4sf7J8Fks5qjMwRgaJpZM4JH5q2.

[marcosnils]

@mcolarusso I've just updated the images, I believe those were outdated. Can you please update and try again?

[mcolarusso]

@marcosnils That seems to have fixed the issue.

[feedtherich]

What is this command "bin/plugin" ? I don't see this anywhere, even after installing "gem install ..."

[feedtherich]

Hello, I tried using the mantika's Docker image and I had this problem:

Pipeline aborted due to error", :exception=>com.amazonaws.AmazonClientException: Unable to execute HTTP request: dynamodb.us-east-1.amazonaws.com, :backtrace=>["com.amazonaws.http.AmazonHttpClient.executeHelper(com/amazonaws/http/AmazonHttpClient.java:618)"

Any idea what's going on here? Thanks!!

[marcosnils]

@feedtherich can u share your config?

[feedtherich]

maybe i'm doing this wrong. i'm new to docker. i am just running this script:

sudo docker run mantika/logstash-dynamodb-streams -e '
input { dynamodb {
        endpoint => "dynamodb.us-east-1.amazonaws.com"
        streams_endpoint => "streams.dynamodb.us-east-1.amazonaws.com"
        view_type => "new_and_old_images"
        aws_access_key_id => "<redacted for security>"
        aws_secret_access_key => "<redacted for security>"
        table_name => "<redacted for security>"
}   }
filter { dynamodb {} }
output { elasticsearch { host => localhost } stdout { } } '

[marcosnils]

@feedtherich if you're using the "latest" image you need to setup the output plugin as follows:

output { elasticsearch { hosts => ["elasticsearch"] } }

note that hosts is pluran and an array of values.

[feedtherich]

Thanks. I still got the same error, however. I think the thing I'm doing wrong is assuming everything is happening automatically without me needing logstash setup locally. Or, am I supposed to do something with one of the two Docker files? I have no idea how to use them.

Thanks.

On Sat, Sep 10, 2016 at 3:43 PM, Marcos Nils notifications@github.com wrote:

@feedtherich https://github.com/feedtherich if you're using the "latest" image you need to setup the output plugin as follows:

output { elasticsearch { hosts => ["elasticsearch"] } }

note that hosts is pluran and an array of values.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/logstash-input-dynamodb/issues/21#issuecomment-246148371, or mute the thread https://github.com/notifications/unsubscribe-auth/AUYcgolTZ-5N5c9zrZzRnFvKkkCCbbzgks5qozKdgaJpZM4JH5q2 .

[fast1234]

I am unable to install this plugin on my EC2 machine(Amazon linux). when I try this command: jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem I am facing this error: ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository Please help

[seriyvolk83]

I may be wrong, but I think the issues are due to postfix "-java" the gems have in the repository. https://rubygems.org/downloads/logstash-core-2.3.1-java.gem.

All other packets look like https://rubygems.org/downloads/dynamodb-1.2.1.gem i.g. nothing goes after version, only .gem.