[Security GenAI] Assistant Anonymization settings shows duplicate available fields when new Kibana Spaces are created

[peluja1012]

Describe the bug: When new Kibana Spaces are created consecutively, the AI Assistant Anonymization settings tab displays duplicate available anonymization fields.

Kibana/Elasticsearch Stack version: v 8.15.0 BC3

Steps to reproduce:

1. Create 2-3 spaces consecutively and quickly by navigating to Stack Management -> Spaces -> Create Space
2. Navigate each individual space. Go to Stack Management -> AI Assistants -> Elastic AI Assistant for Security. Then go to the Anonymization settings tab.
3. Notice that in some of the spaces, the Anonymization settings tab displays duplicated fields (i.e. host.name appears twice).

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[angorayc]

Test env: https://p.elstc.co/paste/TTt5LqFd#snUhCFJSfCU-cHM+srgWhfaZRIrpMpcH6HBAk6AVFPa

The fields duplicated after a new space created and landing on a Security Solution's page. (It fetched 200 fields When landing in the anonymization settings tab without further interaction. I used Devtools to check the count of the index, it did have 200 fields in it.) Therefore, I'm looking into the code where the index is initialised and written.

Current spaces with duplicated fields:

Space s2 with duplicated fields:

Space s2 with No duplicated fields:

[angorayc]

[angorayc]

I was able to reproduce this sometimes after a new space created in https://new-bc3.kb.us-west2.gcp.elastic-cloud.com/ and landing on http://localhost:5601/s/{spaceId}/app/security/get_started.

It is not necessary to land on the AI settings page to have the fields created twice, so it is very likely that there is something unexpected happened when initialising anonymization fields after a new space get created.