augtool: dump-json support?

[examon]

@lutter

Hi, I want to ask if it would be possible to add support for json output. It would help me and my team a lot if there was a dump-json command for augtool in addition to the dump-xml.

Right now, we are transforming XML output that augtool produces to the matching json. The problem is that this transformation takes relatively long time (because of fnmatch, see https://github.com/leapp-to/leapp-actors/blob/master/src/actors/common/augeas/augeas.py#L53).

Thank you

[raphink]

That could be nice indeed. How do you imagine the mapping would go?

[lutter]

I'd be very interested in converting the Augeas tree to JSON. The tricky part is what exactly the JSON representation would look like - it would be easy enough to do a direct translation of the tree into JSON, i.e. each node in the tree would become an object

  { "label": <label>
    "value": <value>
    "children: [ <node>, <node>, ... ] }

I have been hoping to get to a more compact representation, something where each level in the tree becomes a JSON object; e.g., something like the tree at /files/etc/hosts/1 might become

  { "ipaddr": "127.0.0.1",
    "canonical": "localhost",
    "alias": [ "localhost.localdomain", "local.example.com" ] }

but such a representation would lose ordering (since the keys in a JSON object are not ordered) and it's not clear how you deal, e.g., with #comment nodes interspersed with other nodes. It's also hard to produce output that is stable, i.e., where the form of the output is the same for a given file, regardless of what a file contains. As an example, you might turn the options from /etc/fstab into an array of strings and that will work until one of those options has a value. At that point, each option would need to become an object.

Looking at your code, I see some other ways in which it could be sped up:

• switch to using the Python bindings to libaugeas rather than invoking it from the command line; right now, both get_lenses and process_augeas_data run augtool, and have it load everything which can be very slow
• when you use the bindings, you can call aug_to_xml directly, which returns an xmlNode, avoiding parsing the XML text
• you can find what lenses were used for a given file file by doing a get /augeas/files/<file>/lens
• similarly, to find all the files that were loaded with the lens Mod.lns, you'd do match /augeas/files//path[../lens = "Mod.lns" or ../lens = "@Mod"]
• if you want to look at a really fast way to iterate over the tree, have a look at how augmatch does it - it should be possible to emulate that technique in Python, though that might be overkill, and would require the not-yet-released Augeas 1.10 ;)

[examon]

@lutter

it would be easy enough to do a direct translation of the tree into JSON, i.e. each node in the tree would become an object

I think this would be good for my use case.

Basically, the code that I've sent you should be doing direct translation from xml to json. Then, we take that json and filter things that we need.

If augeas would support dump-json, then we could drop our xml2json code, which would be very nice.

[lutter]

Sounds good .. now I (or somebody else) just needs to find the time to implement this ;)

[examon]

Btw, Is there API for dumping xml? I could not find anything. Right now, I'm doing the things I need with API, but then I need call augtool dump-xml. I'm not sure if there is better way.

[lutter]

The API is aug_to_xml (unfortunately, the best API docs are still the header file augeas.h)