Implement single source of truth for each index

[k-allagbe]

[!NOTE]
This is a temporary measure, until our data lake is ready.

Description

Set up a central repository which will be the single source of truth for each index to facilitate automated index updates and improve data reliability.

Tasks

• [ ] Identify existing data sources.
• [ ] Set up a central data repository, most likely unstructured like Blob, for each index.
• [ ] Enforce read and write policies: only system admins should write.
• [ ] Ensure private data stay on private infrastructure.
• [ ] Manually update the indexes with the new source.

Acceptance Criteria

• All indexes are up to date with the sources of truth.
• Repository access is correctly restricted: writable only by admins.
• Security measures are implemented and effective.