[Feature Request] Separation of Merges

[linuxpi]

Is your feature request related to a problem? Please describe

Most of the codebase for Opensearch today exists as a Monolith in :server hosting code related to various background tasks, including Merge. Segment Merges are run from within the Engine and are triggered only on the data node where a particular Shard is assigned.

It would be an anti-pattern to build entire :sever jar and host on Offline Node, which is just responsible for performing Merges.

Current Merge Flow

Once IndexWriter triggers a merge, following Merge Component come into play

Describe the solution you'd like

While maintaining the Status Quo, we need a way to separate out individual components like “Merge” and be able to run separately on Offline Fleet.

Related component

Storage

Describe alternatives you've considered

No response

Additional context

https://github.com/opensearch-project/OpenSearch/issues/12361 https://github.com/opensearch-project/OpenSearch/issues/12725 https://github.com/opensearch-project/opensearch/issues/5910

[andrross]

[Triage - attendees 1 2 3 4 5 6] @linuxpi Thanks, there's not a lot of detail here but I'll go ahead and remove the untriaged label since this looks like something you're tracking in a separate meta issue. In the future, if you'd prefer not to have these issues triaged please remove the label when creating the issue.

[linuxpi]

[Triage - attendees 1 2 3 4 5 6] @linuxpi Thanks, there's not a lot of detail here but I'll go ahead and remove the untriaged label since this looks like something you're tracking in a separate meta issue. In the future, if you'd prefer not to have these issues triaged please remove the label when creating the issue.

My bad, i forgot to remove the label. Thanks for removing it! Also, I've added some details here which was definitely missing earlier. This will get more details as we progress further and bring more clarity after exploration