search

opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.77k stars 1.82k forks source link

[RW Separation] Updates to restore from snapshot with added search replicas #15532

Closed mch2 closed 2 weeks ago

mch2 commented 2 months ago

With https://github.com/opensearch-project/OpenSearch/issues/15368 we add the ability to create and update an index to have search replicas. We need to also support this configuration on restore. With this issue we should accomplish this while honoring the same validations for search replicas:

  1. Search replicas allowed with Segrep only
  2. Auto expand replica count not allowed
  3. Feature flag enforced
vinaykpud commented 1 month ago

Hi @mch2 I would like to pickup this issue

vinaykpud commented 1 month ago

Reproduced this issue and found that:

  1. Search replicas not getting restored from the snapshot
  2. We need to add validations while restoring the snapshot with replication type SEGMENT, so that search replicas do not get created when replication type is DOCUMENT.
vinaykpud commented 1 month ago

Extending above mentioned validation adding a detailed validations bellow and marking which needs to be allowed and and which needs to be blocked.

Scenario 1:

Snapshot created with index.replication.type: DOCUMENT with no search replica (note: Search replicas are allowed only in SEGMENT replication type), can be restored with bellow settings: Case 1. No explicit index.replication.type and index.number_of_search_only_replicas in the settings: Allow Case 2. No explicit index.replication.type and index.number_of_search_only_replicas = 0: Allow Case 3. index.replication.type: DOCUMENT and no explicit index.number_of_search_only_replicas : Allow Case 4. index.replication.type: DOCUMENT and index.number_of_search_only_replicas = 0 : Allow Case 5. index.replication.type: DOCUMENT and index.number_of_search_only_replicas > 0 : Block Case 6. index.replication.type: SEGMENT and no explicit index.number_of_search_only_replicas : Allow Case 7. index.replication.type: SEGMENT and index.number_of_search_only_replicas = 0 : Allow Case 8. index.replication.type: SEGMENT and index.number_of_search_only_replicas > 0 : Allow

Scenario 2:

Snapshot created with index.replication.type: SEGMENT with no search replica. Case 1. No explicit index.replication.type and index.number_of_search_only_replicas in the settings: Allow Case 2. No explicit index.replication.type and index.number_of_search_only_replicas = 0: Allow Case 3. index.replication.type: DOCUMENT and no explicit index.number_of_search_only_replicas : Allow Case 4. index.replication.type: DOCUMENT and index.number_of_search_only_replicas = 0 : Allow Case 5. index.replication.type: DOCUMENT and index.number_of_search_only_replicas > 0 : Block Case 6. index.replication.type: SEGMENT and no explicit index.number_of_search_only_replicas : Allow Case 7. index.replication.type: SEGMENT and index.number_of_search_only_replicas = 0 : Allow Case 8. index.replication.type: SEGMENT and index.number_of_search_only_replicas > 0 : Allow

Scenario 3:

Snapshot created with index.replication.type: SEGMENT with index.number_of_search_only_replicas > 0 Case 1. No explicit index.replication.type and index.number_of_search_only_replicas in the settings: Allow Case 2. No explicit index.replication.type and index.number_of_search_only_replicas = 0: Allow Case 3. index.replication.type: DOCUMENT and no explicit index.number_of_search_only_replicas : Block Case 4. index.replication.type: DOCUMENT and index.number_of_search_only_replicas = 0 : Allow Case 5. index.replication.type: DOCUMENT and index.number_of_search_only_replicas > 0 : Block Case 6. index.replication.type: SEGMENT and no explicit index.number_of_search_only_replicas : Allow Case 7. index.replication.type: SEGMENT and index.number_of_search_only_replicas = 0 : Allow Case 8. index.replication.type: SEGMENT and index.number_of_search_only_replicas > 0 : Allow