search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.21k stars 3.5k forks source link

sync mongodb and elasticsearch by logstash #11919

Open mostaqimi opened 4 years ago

mostaqimi commented 4 years ago

I just want to sync the data from mongodb to elastic using logstash. Its working good, when any new record comes in mongodb, logstash pushes into elastic. But when I update any record in mongodb then it does not change into elasticsearch. I want to make changes in the config file so that when any record updates in mongo it should reflect in elastic as well. I have already tried by making action => "index" as well as action =>"update" in .conf file.

input{ mongodb{ uri => 'mongodb://localhost:27017/DBname'

placeholder_db_dir => '/opt/logstash' placeholder_db_name => 'logstash_sqlite.db' collection => "collectionName" batch_size => 500 } }

filter { mutate { remove_field => [ "_id" ] } }

output{ elasticsearch{ action => "index" hosts => ["localhost:9200"] index => "mongo_hi_log_data" } stdout{ codec => rubydebug } }

lyfuci commented 3 years ago

I have the same doubts when sync data between 2 elasticsearches , any response would be greatly appreciated.