Try YAML based explorers

[Marigold]

Motivation

Our metadata is gradually converging into YAML files within the ETL pipeline, including dataset metadata, multi-dimensional indicators, and TSV files created by create_explorer (though technically stored in a config dictionary). YAML offers several benefits:

• Improved readability
• Schema validation
• Autocompletion via Copilot
• Support for placeholders and templating

It would be an interesting experiment to convert some TSV files into YAML and assess their user-friendliness.

Solution

Design a YAML schema similar to the one used for multi-dimensional indicators (mdims) and attempt to generate a TSV file from it. What are the differences between the mdims and explorer YAML files? If successful, this could be a first step toward unifying mdims and explorers under a YAML-based format.

Open Questions

• Do we gain any advantages from switching to YAML files? Or would it be wiser to wait for a decision on merging explorers and mdims?
• Can authors still make edits easily? (And do they actually do so?)
• How do we preview explorers? (We could use staging servers for previews, similar to how we handle data pages.)

[larsyencken]

Totally agree that YAML is the better format. Perhaps the ideal starting point is a TSV blob in MySQL, which we could then try moving to/from some nicer YAML format.

Researchers still occasionally update explorers in the admin, but we believe it's less often that they create them now.