search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.45k stars 24.58k forks source link

Optimize error handling after lazy rollovers #108838

Open nielsbauman opened 3 months ago

nielsbauman commented 3 months ago

Currently, we're looping over all the requests in the bulk request for each failed lazy rollover (here, and once https://github.com/elastic/elasticsearch/pull/108108 is merged, we'll do it in BulkOperation too). While this is probably around O(n) in practice, it's technically O(n^2), if each document was destined to a different failure store and each rollover failed.

A better alternative would be to collect all the failed rollovers and traverse the documents one time.

elasticsearchmachine commented 3 months ago

Pinging @elastic/es-data-management (Team:Data Management)

cam4phor commented 1 month ago

Hi @nielsbauman, may I work on this issue?

nielsbauman commented 1 month ago

Hi @cam4phor, sure, go ahead! Thanks for your interest in Elasticsearch :)

cam4phor commented 1 month ago

Thanks, I'll update my findings here

vishakha-mali commented 1 month ago

well, I'm new on open source.I have seen this project using java but I'm sure where to start.Can anyone help me...

Kushal-Dalasaniya commented 1 month ago

Hi @nielsbauman , I have been working on optimizing error handling after lazy rollovers and have raised a PR for this: PR #111572. Additionally, I have signed the contributors' document.

As I am new to open source contributions, I would greatly appreciate any feedback or guidance you can provide to ensure I have addressed this issue thoroughly.