Rebuilding elastic index takes a lot of time

OrchardCMS / OrchardCore

Orchard Core is an open-source modular and multi-tenant application framework built with ASP.NET Core, and a content management system (CMS) built on top of that framework.

https://orchardcore.net

BSD 3-Clause "New" or "Revised" License

7.39k stars 2.38k forks source link

Rebuilding elastic index takes a lot of time #16308

Closed konika-jindal closed 2 months ago

konika-jindal commented 3 months ago

I am having almost 25000 data for an elasticsearch index to rebuild , it takes almost 12-14 hours to rebuild , if I want to add just a small change and rebuild or reset then it takes almost 12-14 hours to rebuild . I want to reduce its time . How can I do that?

sebastienros commented 3 months ago

@konika-jindal do you have suggestions to improve it? Have you tried debugging the background indexing task and see what part was taking too much time? For instance could you try to change the batch size to see if it has any effect.

Maybe there is a way to batch the documents that are sent to ES, it could be a way to improve perf. Or maybe other best practices on how to optimize big numbers of documents to ES. Could try to you investigate?

konika-jindal commented 3 months ago

Yes sure , I will try out these ways and let you know for same .

hyzx86 commented 3 months ago

This PR may be related to this issue, but I don't have time to deal with them at the moment.

https://github.com/OrchardCMS/OrchardCore/pull/14063

konika-jindal commented 3 months ago

I am not getting anything in background task as required , is there another way to go?

github-actions[bot] commented 2 months ago

It seems that this issue didn't really move for quite a while despite us asking the author for further feedback. Is this something you'd like to revisit any time soon or should we close? Please reply.

github-actions[bot] commented 2 months ago

Closing this issue because it didn't receive further feedback from the author for very long. If you think this is still relevant, feel free to reopen it with the requested details.