richardwilly98 / elasticsearch-river-mongodb

MongoDB River Plugin for ElasticSearch
1.12k stars 215 forks source link

error occurs when import db with elasticsearch-river-mongo and mongodb 2.6.5 #532

Open jindov opened 9 years ago

jindov commented 9 years ago

This error occur when I run elasticsearch with mongo river, .... [148]: index [mydb], type [all], id [547ebcb217dc1346f1bd5fd9], message [OutOfMemoryError[unable to create new native thread]] [149]: index [mydb], type [all], id [547fceff17dc1346efbd5fe4], message [OutOfMemoryError[unable to create new native thread]] [150]: index [mydb], type [all], id [547fceff17dc1346efbd5fe4], message [OutOfMemoryError[unable to create new native thread]] [151]: index [mydb], type [all], id [547fcf0717dc1346f0bd605a], message [OutOfMemoryError[unable to create new native thread]] [152]: index [mydb], type [all], id [547fcf0717dc1346f0bd605a], message [OutOfMemoryError[unable to create new native thread]] [153]: index [mydb], type [all], id [547fcf0f17dc1346f2bd600b], message [OutOfMemoryError[unable to create new native thread]] [2015-06-03 09:57:44,249][INFO ][river.mongodb.util ] setRiverStatus called with mongodb_all - IMPORT_FAILED [2015-06-03 09:57:44,259][ERROR][org.elasticsearch.river.mongodb.MongoDBRiverBulkProcessor] Bulk processor failed. failure in bulk execution: [0]: index [_river], type [mongodb_all], id [mydb.videos_ver2], message [OutOfMemoryError[unable to create new native thread]] [2015-06-03 09:57:44,279][INFO ][river.mongodb.util ] setRiverStatus called with mongodb_all - IMPORT_FAILED [2015-06-03 09:57:44,278][ERROR][org.elasticsearch.river.mongodb.MongoDBRiverBulkProcessor] Bulk processor failed. failure in bulk execution: [0]: index [_river], type [mongodb_tvshow], id [mydb.videos_ver2], message [OutOfMemoryError[unable to create new native thread]] [2015-06-03 09:57:44,279][INFO ][river.mongodb.util ] setRiverStatus called with mongodb_tvshow - IMPORT_FAILED [2015-06-03 09:57:44,314][ERROR][org.elasticsearch.river.mongodb.MongoDBRiverBulkProcessor] Bulk processor failed. failure in bulk execution: [0]: index [mydb], type [tvshow], id [556dcb7d17dc134317cef426], message [OutOfMemoryError[unable to create new native thread]] [2015-06-03 09:57:44,314][INFO ][river.mongodb.util ] setRiverStatus called with mongodb_tvshow - IMPORT_FAILED [2015-06-03 09:57:44,320][ERROR][org.elasticsearch.river.mongodb.MongoDBRiverBulkProcessor] Bulk processor failed. failure in bulk execution: [0]: index [mydb], type [all], id [556dcb7d17dc134317cef426], message [OutOfMemoryError[unable to create new native thread]] [1]: index [mydb], type [all], id [556dcb7d17dc134317cef426], message [OutOfMemoryError[unable to create new native thread]] [2015-06-03 09:58:46,353][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Closing river mongodb_tvshow [2015-06-03 09:58:46,465][INFO ][org.elasticsearch.river.mongodb.Slurper] river-mongodb slurper interrupted [2015-06-03 09:58:46,465][INFO ][org.elasticsearch.river.mongodb.Slurper] river-mongodb slurper interrupted [2015-06-03 09:58:46,468][INFO ][org.elasticsearch.river.mongodb.Slurper] Slurper is stopping. River has status STOPPED [2015-06-03 09:58:46,468][INFO ][org.elasticsearch.river.mongodb.Slurper] Slurper is stopping. River has status STOPPED [2015-06-03 09:59:43,378][DEBUG][action.bulk ] [Abyss] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m] [2015-06-03 09:59:43,379][ERROR][org.elasticsearch.river.mongodb.MongoDBRiverBulkProcessor] Bulk processor failed. failure in bulk execution: [0]: index [mydb], type [all], id [5530b8d117dc135c8bd7d7e5], message [UnavailableShardsException[[mydb][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@332dd4cc]] [1]: index [mydb], type [all], id [5530b8d117dc135c8bd7d7e5], message [UnavailableShardsException[[mydb][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@332dd4cc]]

Anyone who has an idea for this error. I've increased bulk size to 2000 an queue to 10000 but it's not work fine