logstash_sqlite.db problems

[pdaniel]

I have a problem when i configure the plugin. I've created a config file :

input { mongodb { uri => 'mongodb://mongourl/mng?ssl=false' placeholder_db_dir => '/opt/logstash-mongodb/' placeholder_db_name => 'logstash_sqlite.db' collection => 'pages_feed' batch_size => 5000 } }

filter {

}

output { elasticsearch { hosts => ["elasticurl:9200"] index => "pagesfeed" document_type => "feed" document_id => "%{id}" } stdout { codec => rubydebug } }

When logstash executes the file i get the error :

{ : timestamp=>"2016-09-06T09:10:32.424000+0000", : message=>"An unexpected error occurred!", : error=>#<Sequel: : DatabaseConnectionError: Java: : JavaSql: : SQLException: pathto'/opt/logstash-mongodb/logstash_sqlite.db': '/opt/logstash-mongodb'doesnotexist>, : class=>"Sequel::DatabaseConnectionError", : backtrace=>["org.sqlite.core.CoreConnection.open(org/sqlite/core/CoreConnection.java:190)", "org.sqlite.core.CoreConnection.(org/sqlite/core/CoreConnection.java:74)", "org.sqlite.jdbc3.JDBC3Connection.(org/sqlite/jdbc3/JDBC3Connection.java:24)", "org.sqlite.jdbc4.JDBC4Connection.(org/sqlite/jdbc4/JDBC4Connection.java:23)", "org.sqlite.SQLiteConnection.(org/sqlite/SQLiteConnection.java:45)", "org.sqlite.JDBC.createConnection(org/sqlite/JDBC.java:114)", "org.sqlite.JDBC.connect(org/sqlite/JDBC.java:88)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.connect(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/adapters/jdbc.rb:222)", "RUBY.make_new(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool.rb:103)", "RUBY.make_new(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:224)", "RUBY.available(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:197)", "RUBY._acquire(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:133)", "RUBY.acquire(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:147)", "RUBY.sync(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:265)", "org.jruby.ext.thread.Mutex.synchronize(org/jruby/ext/thread/Mutex.java:149)", "RUBY.sync(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:265)", "RUBY.acquire(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:146)", "RUBY.hold(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/connection_pool/threaded.rb:104)", "RUBY.synchronize(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/database/connecting.rb:256)", "RUBY.execute(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/adapters/jdbc.rb:247)", "RUBY.execute(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:952)", "RUBY.fetch_rows(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/adapters/jdbc.rb:738)", "RUBY.with_sql_each(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:835)", "RUBY.with_sql_first(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:843)", "RUBY.single_record!(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:661)", "RUBY.single_record(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:649)", "RUBY.first(/opt/logstash/vendor/bundle/jruby/1.9/gems/sequel-4.33.0/lib/sequel/dataset/actions.rb:200)", "RUBY.", "RUBY.get_placeholder(/opt/logstash/vendor/local_gems/9a1942b2/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:98)", "RUBY.update_watched_collections(/opt/logstash/vendor/local_gems/9a1942b2/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:146)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.update_watched_collections(/opt/logstash/vendor/local_gems/9a1942b2/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:144)", "RUBY.register(/opt/logstash/vendor/local_gems/9a1942b2/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:168)", "RUBY.start_inputs(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:322)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.start_inputs(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:321)", "RUBY.start_workers(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:172)", "RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:126)", "RUBY.execute(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/agent.rb:210)", "RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/runner.rb:90)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/runner.rb:95)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.initialize(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24)", "java.lang.Thread.run(java/lang/Thread.java:745)"], : level=>: warn }

My problem is with logstash_sqlite.db. The database should exist?

[harshit1810]

you can comment out the line with placeholder_db_name .....logstash automatically creates the logstash_sqlite.db file

[RamVarma89]

I have commented out that particular line but am still getting the errors

Error: Pipeline aborted due to error :exception=>"Sequel::DatabaseConnectionError",

[alexsnti]

Same problem here. Any clue ?

[alexsnti]

I found a solution for my problem.

In the example config file, the placeholder_db_dir is set as '/opt/logstash-mongodb/'.

I changed for '/opt/logstash' and it worked. It was just a permission issue.

[f4ct0r]

As @alexsnti said. First, you should guarantee the directory '/opt/logstash-mongodb/' exists, then you should make sure the user 'logstash' has permission to write in the directory

[Alansdfs]

On Windows I change the "placeholder_db_dir" route to "C://path" :D

[vivs0]

On linux you need to first create the directories & change its ownership to logstash. chown logstash:logstash /opt/ and restart the logstash

[boggumaheshbabu]

I found a solution for my problem.

In the example config file, the placeholder_db_dir is set as '/opt/logstash-mongodb/'.

I changed for '/opt/logstash' and it worked. It was just a permission issue.

@alexsnti thanks for this. It worked.