search

phutchins / logstash-input-mongodb

MongoDB input plugin for Logstash
Other
187 stars 104 forks source link

More than one collection as input #88

Open ghost opened 6 years ago

ghost commented 6 years ago

Hi there, I'm trying to get data from two collections "logs" and "servers_output" from a mongodb database and insert it into an ES.

I works for one collection but when I try to use two collection it doesn't work.

The documentation says that is an string, "A regex that will be used to find desired collecitons."

So this is my cfg : 

mongodb {
    uri => 'mongodb://xxxx:yyyyyy@1.2.3.4/pritunl?ssl=false'
    placeholder_db_dir => '/opt/logstash-mongodb/'
    placeholder_db_name => 'servers_output.db'
    collection => '(logs,servers_output)'
    batch_size => 5000
 }

But i get this error: 

{:timestamp=>"2018-01-04T15:51:33.138000+0000", :message=>"Pipeline aborted due to error", :exception=>"NoMethodError", :error=>"undefined method `[]' for nil:NilClass", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:89:in `init_placeholder'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:99:in `get_placeholder'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:146:in `update_watched_collections'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:144:in `update_watched_collections'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:168:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:330:in `start_inputs'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:329:in `start_inputs'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:180:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/agent.rb:491:in `start_pipeline'"], :level=>:error}
{:timestamp=>"2018-01-04T15:51:36.145000+0000", :message=>"stopping pipeline", :id=>"main"}

==> /var/log/logstash/logstash.stdout <==
{:timestamp=>"2018-01-04T15:51:36.145000+0000", :message=>"stopping pipeline", :id=>"main"}

Thanks in advance.

191801737 commented 6 years ago

mongodb { uri => 'mongodb://xxxx:yyyyyy@1.2.3.4/pritunl?ssl=false' placeholder_db_dir => '/opt/logstash-mongodb/' placeholder_db_name => 'servers_output.db' collection => 'logs' batch_size => 5000 } mongodb { uri => 'mongodb://xxxx:yyyyyy@1.2.3.4/pritunl?ssl=false' placeholder_db_dir => '/opt/logstash-mongodb/' placeholder_db_name => 'servers_output.db' collection => 'servers_output' batch_size => 5000 }

try this

zpol commented 6 years ago

It worked! thanks, I'm also using:

(collection1|collection2)