search

vstakhov / libucl

Universal configuration library parser
BSD 2-Clause "Simplified" License
1.63k stars 139 forks source link

readme

UCL - Universal Configuration Language

Table of Contents generated with DocToc

Introduction

This repository provides the C library for parsing configurations written in UCL - universal configuration language. It also provides functions to operate with other formats:

If you are looking for the libucl API documentation you can find it at this page.

Basic structure

UCL is heavily infused by nginx configuration as the example of a convenient configuration system. However, UCL is fully compatible with JSON format and is able to parse json files. For example, you can write the same configuration in the following ways:

param = value;
section {
    param = value;
    param1 = value1;
    flag = true;
    number = 10k;
    time = 0.2s;
    string = "something";
    subsection {
        host = {
            host = "hostname";
            port = 900;
        }
        host = {
            host = "hostname";
            port = 901;
        }
    }
}
{
    "param": "value",
    "section": {
        "param": "value",
        "param1": "value1",
        "flag": true,
        "number": 10000,
        "time": "0.2s",
        "string": "something",
        "subsection": {
            "host": [
                {
                    "host": "hostname",
                    "port": 900
                },
                {
                    "host": "hostname",
                    "port": 901
                }
            ]
        }
    }
}

Improvements to the json notation.

There are various things that make ucl configuration more convenient for editing than strict json:

General syntax sugar

"key": "value"

is equal to:

{"key": "value"}
key = value;
section {
    key = value;
}

is equal to:

{
    "key": "value",
    "section": {
        "key": "value"
    }
}
{
    "key1": "value",
    "key2": "value",
}

Automatic arrays creation

{
    "key": "value1",
    "key": "value2"
}

is converted to:

{
    "key": ["value1", "value2"]
}

Named keys hierarchy

UCL accepts named keys and organize them into objects hierarchy internally. Here is an example of this process:

section "blah" {
    key = value;
}
section foo {
    key = value;
}

is converted to the following object:

section {
    blah {
        key = value;
    }
    foo {
        key = value;
    }
}

Plain definitions may be more complex and contain more than a single level of nested objects:

section "blah" "foo" {
    key = value;
}

is presented as:

section {
    blah {
        foo {
            key = value;
        }
    }
}

Convenient numbers and booleans

General improvements

Comments

UCL supports different style of comments:

Multiline comments may be nested:

# Sample single line comment
/*
 some comment
 /* nested comment */
 end of comment
*/

Macros support

UCL supports external macros both multiline and single line ones:

.macro_name "sometext";
.macro_name {
    Some long text
    ....
};

Moreover, each macro can accept an optional list of arguments in braces. These arguments themselves are the UCL object that is parsed and passed to a macro as options:

.macro_name(param=value) "something";
.macro_name(param={key=value}) "something";
.macro_name(.include "params.conf") "something";
.macro_name(#this is multiline macro
param = [value1, value2]) "something";
.macro_name(key="()") "something";

UCL also provide a convenient include macro to load content from another files to the current UCL object. This macro accepts either path to file:

.include "/full/path.conf"
.include "./relative/path.conf"
.include "${CURDIR}/path.conf"

or URL (if ucl is built with url support provided by either libcurl or libfetch):

.include "http://example.com/file.conf"

.include macro supports a set of options:

Priorities are used by UCL parser to manage the policy of objects rewriting during including other files as following:

By default, the priority of top-level object is set to zero (lowest priority). Currently, you can define up to 16 priorities (from 0 to 15). Includes with bigger priorities will rewrite keys from the objects with lower priorities as specified by the policy. The priority of the top-level or any other object can be changed with the .priority macro, which has no options and takes the new priority:

# Default priority: 0.
foo = 6
.priority 5
# The following will have priority 5.
bar = 6
baz = 7
# The following will be included with a priority of 3, 5, and 6 respectively.
.include(priority=3) "path.conf"
.include(priority=5) "equivalent-path.conf"
.include(priority=6) "highpriority-path.conf"

Variables support

UCL supports variables in input. Variables are registered by a user of the UCL parser and can be presented in the following forms:

UCL currently does not support nested variables. To escape variables one could use double dollar signs:

However, if no valid variables are found in a string, no expansion will be performed (and $$ thus remains unchanged). This may be a subject to change in future libucl releases.

Multiline strings

UCL can handle multiline strings as well as single line ones. It uses shell/perl like notation for such objects:

key = <<EOD
some text
splitted to
lines
EOD

In this example key will be interpreted as the following string: some text\nsplitted to\nlines. Here are some rules for this syntax:

key <<EOD

some
text

EOD

Single quoted strings

It is possible to use single quoted strings to simplify escaping rules. All values passed in single quoted strings are NOT escaped, with two exceptions: a single ' character just before \ character, and a newline character just after \ character that is ignored.

key = 'value'; # Read as value
key = 'value\n\'; # Read as  value\n\
key = 'value\''; # Read as value'
key = 'value\
bla'; # Read as valuebla

Emitter

Each UCL object can be serialized to one of the four supported formats:

Validation

UCL allows validation of objects. It uses the same schema that is used for json: json schema v4. UCL supports the full set of json schema with the exception of remote references. This feature is unlikely useful for configuration objects. Of course, a schema definition can be in UCL format instead of JSON that simplifies schemas writing. Moreover, since UCL supports multiple values for keys in an object it is possible to specify generic integer constraints maxValues and minValues to define the limits of values count in a single key. UCL currently is not absolutely strict about validation schemas themselves, therefore UCL users should supply valid schemas (as it is defined in json-schema draft v4) to ensure that the input objects are validated properly.

Performance

Are UCL parser and emitter fast enough? Well, there are some numbers. I got a 19Mb file that consist of ~700 thousand lines of json (obtained via http://www.json-generator.com/). Then I checked jansson library that performs json parsing and emitting and compared it with UCL. Here are results:

jansson: parsed json in 1.3899 seconds
jansson: emitted object in 0.2609 seconds

ucl: parsed input in 0.6649 seconds
ucl: emitted config in 0.2423 seconds
ucl: emitted json in 0.2329 seconds
ucl: emitted compact json in 0.1811 seconds
ucl: emitted yaml in 0.2489 seconds

So far, UCL seems to be significantly faster than jansson on parsing and slightly faster on emitting. Moreover, UCL compiled with optimizations (-O3) performs significantly faster:

ucl: parsed input in 0.3002 seconds
ucl: emitted config in 0.1174 seconds
ucl: emitted json in 0.1174 seconds
ucl: emitted compact json in 0.0991 seconds
ucl: emitted yaml in 0.1354 seconds

You can do your own benchmarks by running make check in libucl top directory.

Conclusion

UCL has clear design that should be very convenient for reading and writing. At the same time it is compatible with JSON language and therefore can be used as a simple JSON parser. Macro logic provides an ability to extend configuration language (for example by including some lua code) and comments allow to disable or enable the parts of a configuration quickly.