Multiplatform config file parsing

[westnordost]

In various places, the app loads and parses some simple YAML files for configuration:

• [x] a dictionary of version number to html fragments that are the changelog (changelog.yml) → #5439
• a dictionary of street name abbreviations with their expansions (abbreviations.yml)
• a list of main contributors (credits_main.yml)
• a list of side project contributors (credits_projects.yml)
• a list of all code contributors on github (credits_contributors.yml)
• a list of all art contributors (credits_art.yml)
• a dictionary of languages -> translator name -> number of translated words (credits_translators.yml)
• a dictionary of flag alignments by country code (flag_alignments.yml)
• a list of all available app translations (languages.yml)
• a dictionary of various country properties (<country code>.yml)
• a dictionary of ATM operators per country (atmOperators.yml)
• a dictionary of charging station operators per country (chargingStationOperators.yml)
• a dictionary of clothes container operators per country (clothesContainerOperators.yml)
• a dictionary of most produced/cultivated crops/fruits per country (orchardProduces.yml)

keep YAML

Currently, com.charleskorn.kaml is used which is a plugin for kotlinx-serialization, however, no native targets are supported currently and noone is working on it at the time of writing (see https://github.com/charleskorn/kaml/issues/232).

Another kotlin multiplatform yaml plugin exists, net.mamoe.yamlkt, but according to the readme, it is (still) in alpha state. Multiline strings, which are currently used in changelog.yml, are "partially supported" since v0.13.0, not sure if this means whether the way multiline strings are used in this project is supported. (changelog.yml could be changed to be a pure HTML file, though, or one file for every version.)

If none of these libraries have proper multiplatform native support at the time this ticket is tackled, the task is to convert the listed files to another config file format like JSON or TOML and parse them as these files instead.

Some of these files are generated automatically, such as credits_contributors.yml, so some build scripts need to be adapted, too. Below some notes regarding each:

convert to TOML

• + well-human-readable format, easy to edit

• - kotlinx-serialization plugin com.akuleshov7-ktoml-core is multiplatform but implementation is incomplete. In particular, "Array of Tables" feature is missing as the time of writing, which is necessary for credits_translators.yml

• + kotlinx-serialization plugin net.peanuuutz.tomlkt is multiplatform and is actually complete, looks well tested and has a useful interface. It's rather fresh at the time of writing, though (meaning, we don't know how well tested it is in production and how committed the author is to maintain it).

• - the above is only marginally smaller (250KB) as net.mamoe.yamlkt (261KB) even though one would think that TOML is a much simpler markup language as YAML - maybe that isn't true

convert to JSON

• + less dependencies: kotlinx-serialization-json already is a dependency

• - no comments possible. So, explanatory comments / docs cannot be included in these files

• still well enough human-readable, given that these files are not that often changed (changelog.yml could be changed to be a html file)

• see draft PR: #5444

---

In general, at the time of writing, I think the best way forward is to simply exchange kaml with yamlkt and if this does not work, don't bother with TOML because implementation is equally "alpha" than the YAML plugins but use JSON for everything.

[matkoniecz]

convert to JSON

it may be minor thing, but lack of trailing commas is also highly irritating (one more trap, less clear diffs)

• no comments possible. So, explanatory comments / docs cannot be included in these files

no proper comment support, but can be done with embedding comment field in data structure and similar hacks if really needed

[HolgerJeromin]

You could use json5 (https://json5.org/) which is supported https://github.com/Kotlin/kotlinx.serialization/blob/master/formats/README.md#JSON5

This is "JSON5 – JSON for Humans" and allows single trailing comma and comments and other useful stuff.

[Peanuuutz]

the above is only marginally smaller (250KB) as net.mamoe.yamlkt (261KB)...

This is because:

1. tomlkt has way more side features than just being a plugin for kotlinx.serialization, especially TomlElement part, which we strive to provide more friendly DX than other implementations, and this takes a large proportion of the package size. We could, however, save some space by reducing unnecessary abstraction within internal.
2. yamlkt is not completely done on its own yet, meaning it could be more than that.

Despite tomlkt has less stars than ktoml, it solves all the issues listed in ktoml, which we could say it's more mature and ready for production (actually, in the benchmark directory, we've tested decoding large cargo files from rust community, and only tomlkt succeed when the file is fully as is, compared to ktoml, toml4j, tomlj, night-config, and the speed is good enough). We consider the main part of tomlkt is complete, so feel free to open issues if found any bug or needed some specific features. We're fully devoted on it.

[orchestr7]

Do not think about selecting between different formats: JSON, TOML or YAML, just use Kotlinx.serialization and plugin-in particular serializer based on personal preferences 😄

[westnordost]

kaml will be fully multiplatform in its next release.