Failed to find unique document using index pattern While document exists

[tlierdotfr]

Hi,

Monstache does not find a document while a search request correctly find it, so deletion fails.

I am using monstache recent Tag : 39b8a2503070e04135a64ad805ae957aed29fa18 (in order to use aws WebIdentityToken connection to OpenSearch).

Using document _id : 64803d65cc13bc001b551ef6 as and example, here are requests relative to it : First request seems to be the bulk upload

TRACE 2023/06/07 08:18:46 POST /_bulk HTTP/1.1
Host: opensearch.tools.vitacloud.fr:443
User-Agent: elastic/7.0.31 (linux-amd64)
Content-Length: 7625
Accept: application/json
Content-Type: application/x-ndjson
Accept-Encoding: gzip

{"index":{"_index":"pio-2023","_id":"64803c631031ab001c91b956","version":7241855687632879618,"version_type":"external"}}
{"createdAt":1686125667762, ... <DOC_CONTENT> ... }
{"index":{"_index":"pio-2023","_id":"64803d62cc13bc001b551ef5","version":7241855691927846915,"version_type":"external"}}
{"createdAt":1686125922247, ... <DOC_CONTENT> ... }
{"index":{"_index":"pio-2023","_id":"64803d64adc727001bbea046","version":7241855700517781507,"version_type":"external"}}
{"createdAt":1686125924325, ... <DOC_CONTENT> ... }
{"index":{"_index":"pio-2023","_id":"64803d65cc13bc001b551ef6","version":7241855704812748802,"version_type":"external"}}
{"createdAt":1686125925269, ... <DOC_CONTENT> ... }
{"index":{"_index":"pio-2023","_id":"64803d65cc13bc001b551ef9","version":7241855704812748807,"version_type":"external"}}
{"createdAt":1686125925616, ... <DOC_CONTENT> ... }

And the HTTP 200 response :

TRACE 2023/06/07 08:18:46 HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
Date: Wed, 07 Jun 2023 08:18:46 GMT
Vary: Accept-Encoding, User-Agent

{"took":10,"errors":false,"items":[{"index":{"_index":"pio-2023","_id":"64803c631031ab001c91b956","_version":7241855687632879618,"result":"updated","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2609704,"_primary_term":1,"status":200}},{"index":{"_index":"pio-2023","_id":"64803d62cc13bc001b551ef5","_version":7241855691927846915,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2609705,"_primary_term":1,"status":201}},{"index":{"_index":"pio-2023","_id":"64803d64adc727001bbea046","_version":7241855700517781507,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2609706,"_primary_term":1,"status":201}},{"index":{"_index":"pio-2023","_id":"64803d65cc13bc001b551ef6","_version":7241855704812748802,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2609707,"_primary_term":1,"status":201}},{"index":{"_index":"pio-2023","_id":"64803d65cc13bc001b551ef9","_version":7241855704812748807,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":2609708,"_primary_term":1,"status":201}}]}

Then, just after, the search request :

TRACE 2023/06/07 08:18:47 POST /pio-%2A/_search HTTP/1.1
Host: opensearch.tools.vitacloud.fr:443
User-Agent: elastic/7.0.31 (linux-amd64)
Transfer-Encoding: chunked
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

4e
{"_source":false,"query":{"term":{"_id":"64803d65cc13bc001b551ef6"}},"size":1}
0

And the associated answer with no result :

TRACE 2023/06/07 08:18:47 HTTP/1.1 200 OK
Content-Length: 163
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/json; charset=UTF-8
Date: Wed, 07 Jun 2023 08:18:47 GMT

{"took":10,"timed_out":false,"_shards":{"total":20,"successful":20,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}

And finaly the error log :

ERROR 2023/06/07 08:18:47 Failed to find unique document 64803d65cc13bc001b551ef6 for deletion using index pattern pio-*

It seems like the deletion is sent just after document creation and OpenSearch does not find it. Maybe it has no time to index it ?

I would need a little help on that one.

Thanks