rwynn / monstache-site

documentation for the monstache project
https://rwynn.github.io/monstache-site/
MIT License
22 stars 30 forks source link

Lots of deletes messages while running the command #19

Open pratikgcet opened 4 years ago

pratikgcet commented 4 years ago

I am running monstache -f config.toml and see lot of these messages with deletes. The number of deletes is quite high

INFO 2020/07/04 15:40:49 Resuming stream 'axes_db.inbound_aggregate' from collec tion monstache.tokens using resume name 'default' STATS 2020/07/04 15:41:21 {"Flushed":20,"Committed":42,"Indexed":0,"Created":0," Updated":0,"Deleted":657285,"Succeeded":0,"Failed":657285,"Workers":[{"Queued":0 ,"LastDuration":361000000},{"Queued":0,"LastDuration":296000000},{"Queued":0,"La stDuration":280000000},{"Queued":0,"LastDuration":265000000}]} STATS 2020/07/04 15:41:50 {"Flushed":36,"Committed":73,"Indexed":0,"Created":0," Updated":0,"Deleted":1254725,"Succeeded":0,"Failed":1254725,"Workers":[{"Queued" :0,"LastDuration":103000000},{"Queued":0,"LastDuration":231000000},{"Queued":0," LastDuration":228000000},{"Queued":0,"LastDuration":240000000}]} STATS 2020/07/04 15:42:22 {"Flushed":55,"Committed":113,"Indexed":0,"Created":0, "Updated":0,"Deleted":1911909,"Succeeded":0,"Failed":1911909,"Workers":[{"Queued ":0,"LastDuration":397000000},{"Queued":0,"LastDuration":254000000},{"Queued":0, "LastDuration":62000000},{"Queued":0,"LastDuration":519000000}]}

My config.TOML is mongo-url = "mongodb://pslex01696.gva.pictet.com:27017" elasticsearch-urls = ["http://islex01364:9200"] change-stream-namespaces = ["axes_db.inbound_aggregate"] namespace-regex = '^axes_db.inbound_aggregate$' gzip = true stats = true dropped-collections = true dropped-databases = true replay = false resume = true resume-write-unsafe = false resume-name = "default" resume-strategy = 1 exit-after-direct-reads = false [[mapping]] namespace = "axes_db.inbound_aggregate" index = "axes_db"

ScubaDrew commented 3 years ago

Why are there so many failures when using workers? We too have a high failure count