Delete not working

[cbraunVBM]

Hi,

deletions of documents are not synced to ES with our configuration. Our goal ist creating an individual index for each month based on the document's timestamp. Resulting in following indices:

infinity.white_bots-2020-09 infinity.white_bots-2020-10 infinity.white_bots-2020-11

Additionally we created an alias "infinity.white_bots" . This alias-Index has no write-index because the index must be chosen by the document's timestamp. So far everything is working fine.

When a document is deleted in MongoDB, I can see in the monstache log, that the document is found for deletion, but the document's index is always the alias index "infinity.white_bots", which cannot perform delete-operations. Actually the index should be infinity.white_bots-2020-11

I added delete-strategy = 0 and delete-index-pattern = "infinity.white_bots-*" to address this issue, but both configuration options don't seem to have any effect. Am I missing something in my config?

Monstache Version is 6.7.0 MongoDB: 4.2.2 ES: 7.4.1

I am using this config:

mongo-url = "mongodb://*****@mongodb/?replicaSet=infinity&readPreference=Primary" elasticsearch-urls = ["http://elasticsearch-master:9200" ] change-stream-namespaces = ["infinity.white_bots"] direct-read-namespaces = ["infinity.white_bots"] direct-read-stateful = true resume-name = "white-bots" resume = true gzip = true stats = true index-stats = false dropped-collections = false dropped-databases = false index-oplog-time = true

oplog-ts-field-name = "oplog_ts_new"

delete-strategy = 0 delete-index-pattern = "infinity.white_bots-*"

[[mapping]] namespace = "infinity.white_bots" index = "infinity.white_bots"

[[script]] script = """ module.exports = function(doc) { outer_doc = {}; outer_doc.doc = doc; return outer_doc; } """

[[script]] namespace = "infinity.white_bots" script = """ module.exports = function(doc) { var meta = { index: 'infinity.white_bots'}; var date = new Date(doc.doc.timestamp); var year = date.getFullYear(); var month = ('0' + (date.getMonth() + 1)).slice(-2); meta.index = meta.index + '-' + year + '-' + month; doc._meta_monstache = meta; return doc; } """

[rwynn]

Hi @cbraunVBM , can you add this to your config and give it a shot?

routing-namespaces = [ "" ]

That tells monstache to search for the document on a delete instead of using the default index mapping.

[cbraunVBM]

Hi @rwynn , thank you very much! It is working now.