RFC: TOML parser library

TOML is a pretty nice language that I really like using for configuration files. It aims for human-readability so it is easy to write a config file "by hand". I haven't seen any discussions about integrating a TOML parser into Zephyr RTOS so I'd like to start one.

My tomlc99 integration

Recently I've tried integrating tomlc99 as a Zephyr library. You can check it out here. However tomlc99 makes use of heap which I tried to avoid by implementing a "heap buffer" that you have to provide for the parser. I got it to work but I cannot decide whether this is a good idea. Another downside of my implementation is that you can only have one parser instance at a time but this can be solved by implementing some sort of parser context struct that would contain heap buffer pointer and size.

Alternative

One alternative that i thought of is implementing it in the same style as JSON parser. However I can imagine that 64 bits for checking if fields are present can be quite limiting for a configuration file. In addition to that there are severe limitations when it comes to nested objects which can be important for configuration files.

I have a few ideas on how to expand the object description system for TOML:

Additional fields

Since TOML is mostly used for configuration files i think that it would be beneficial to expand the object definition system used in the JSON library with additional fields like optional and _defaultvalue that would enable users to define default values.

datum objects

Instead of using a 64 bit integer to check which fields were present I'd like to propose defining a output structure that is returned by the parser which uses datum objects instead of values. Example structure of such object:

typedef struct {
  bool ok;
  bool default; /* true if default value was used */
  union {
    int i;
    bool b;
    const char *s; /* alternatively char[] */
    void *obj;
  } u;
} toml_datum_t;

Example

Defining a config file would look like:

typedef struct {
  toml_datum_t a;
  toml_datum_t b;
  toml_datum_t c;
  toml_datum_t section;
} my_config_t;

/* Description of fields */
static const struct toml_obj_descr my_config_def[] = {
  TOML_FIELD_DESCR_INT(my_config_t, a);         /* Required integer field */
  TOML_FIELD_DESCR_INT_OPT(my_config_t, b, 0);  /* Optional integer field with 0 as default value */
  TOML_FIELD_DESCR_STR(my_config_t, c, 12);     /* Required string field with maximum length of 12 */
  TOML_FIELD_DESCR_OBJ(my_config_t, section, my_config_section_t)  /* Required nested object. described by my_config_section_t  */
};

Accessing a value with datum objects would look like:

const char *str = "[section]\n"
                  "foo = \"foo\"\n"

my_config_t cfg;
parse_toml(str, my_config_def, my_config_def_size &cfg);
if(cfg.section.ok) {
  my_config_section_t *section = (my_config_section_t *)cfg.section.u.o;
  if(section.foo.ok) {
    LOG_INF("Value: %s", section.foo.u.s);
  }
}

Generator script

Ok i know that this might be overkill but I thought it could be nice if there was a helper generator script that would automatically define structs, descriptions and maybe some helper definitions for parsing the config files. The developer would provide path(s) to file(s) with TOML object definitions (possibly in json schema) as a Kconfig option and the script would generate neccesary files.

Summary

Let me know if there is interest in such a library. I'd like to start working on it but i want to know if there is interest and how do you feel about my ideas about the parser.

zephyrproject-rtos / zephyr