Proactively improve robustness of FTR tests that use esArchiver

[gsoldevila]

There are multiple FTR tests that still rely on esArchiver to import test data into ES indices.

Some of the archives used by these tests are actually re-defining SO indices and documents.

When esArchiver.load(...) is called in the beginning of a test (usually in a before hook), multiple things happen:

• esArchiver detects that SO indices are being defined
• It deletes the official SO indices created by Kibana.
• It recreates the indices from the definitions present in the mappings.json.
• It inserts the documents defined in the data.json.
• It calls the KibanaMigrator, hoping to update the documents present in the indices.

All of that happens whilst Kibana is already running, and potentially issuing requests against these indices. Sometimes these requests happen right when esArchiver is re-creating the SO indices, causing different sorts of flakiness (as seen on https://github.com/elastic/kibana/issues/158918).

The aforementioned issue has been tackled in 2 stages: 1) Updating esArchiver so that it cleans up the SO indices whenever it detects that SO are defined in the data.json (without actually deleting the indices). (https://github.com/elastic/kibana/pull/159582) 2) Updating the archives related to the flaky tests as follows: (https://github.com/elastic/kibana/pull/159397)

• Deleting the mappings.json. This way, esArchiver will NOT delete + recreate the SO indices.
• Updating the documents in data.json to make sure they are aligned with the current version indices.

In parallel, the esArchiver.load(...) was updated so that when it actually creates SO indices (for those tests that still define them in mappings.json), it does so in two separate steps: first create indices, and then create aliases. This helps circumvent some of the _no_shardserrors that happen when the index is still not ready to be queried. (https://github.com/elastic/kibana/pull/160584)

What's next ATM there are 7 other archives that are still defining "dynamic" SO indices, i.e. indices that aim at being at the latest version, by using the $KIBANA_PACKAGE_VERSION in the mappings.json. These should be easy to update, removing the corresponding definitions from the mappings.json, and ensuring they will not cause flakiness.

Also ATM, there are a total of 49 mappings.json files that presumably define SO indices (they contain the .kibana keyword). Even though they are not dynamic, they could also be updated to not cause SO indices to be deleted. But in order to do so, we must:

• Delete the SO index definitions from the mappings.json files (eventually deleting the files completely if they don't define other indices).
• Update the data.json to place each document in the right index.
• Update the data.json to make sure typeMigrationVersion and coreMigrationVersion are properly defined, and they match the state/structure of each document.

The goal of this issue is to proactively tackle the tests that are still defining SO indices, in order to:

• Avoid deleting + recreating SO indices with Kibana already running.
• Reuse official Kibana indices. More often than not, the indices defined in mappings.json don't closely mirror the structure of the official ones:
  • Sometimes they don't define mappings.
  • Other times they are lacking _meta fields.
  • They allow unknown objects to be inserted (dynamic: false instead of 'strict').

  • They don't account for the .kibana split.

    Dynamic esArchiver archives:

• x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces
  • https://github.com/elastic/kibana/pull/161852
• test/api_integration/fixtures/es_archiver/saved_objects/delete_unknown_types
  • https://github.com/elastic/kibana/pull/161936
• test/functional/fixtures/es_archiver/deprecations_service
  • https://github.com/elastic/kibana/pull/161937
• test/functional/fixtures/es_archiver/saved_objects_management/export_transform TODO
• test/functional/fixtures/es_archiver/saved_objects_management/hidden_saved_objects TODO
• test/functional/fixtures/es_archiver/saved_objects_management/nested_export_transform TODO
• test/functional/fixtures/es_archiver/saved_objects_management/visible_in_management TODO

### Tasks
- [ ] https://github.com/elastic/kibana/issues/168973
- [ ] https://github.com/elastic/kibana/issues/168969
- [ ] https://github.com/elastic/kibana/issues/168926
- [ ] https://github.com/elastic/kibana/issues/169075

[gsoldevila]

I created a script to locally update the data.json entries, leveraging the logic used by esArchiver to parse NDJSON.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/appex-qa (Team:QA)