saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14k stars 5.47k forks source link

[FEATURE REQUEST] JC renderer to convert command output to objects #58355

Open kellyjonbrazil opened 3 years ago

kellyjonbrazil commented 3 years ago

I recently created an Ansible filter plugin for JC (https://github.com/kellyjonbrazil/jc - I am the author) and it was requested I create something similar for Salt. I believe this would be a renderer serializer (deserializer, to be specific), and an outputter in Salt, though I'm open to other options.

JC is a command-line utility and python library that converts the output of dozens of commands and file-types into a python dict or list of dicts. The JC CLI converts the command output to JSON. Here is a blog post that demonstrates how it can be used in Ansible: https://blog.kellybrazil.com/2020/08/30/parsing-command-output-in-ansible-with-jc

I'm still getting my Salt dev environment set up, but I noticed in the Contributing docs it recommended opening an issue before issuing a PR.

~~I have created a first draft of the renderer here: https://github.com/kellyjonbrazil/salt/blob/jc-renderer/salt/renderers/jc.py~~

I have created a first draft of the serializer and outputter here: https://github.com/kellyjonbrazil/salt/blob/jc-renderer/salt/serializers/jc.py https://github.com/kellyjonbrazil/salt/blob/jc-renderer/salt/output/jc.py

~~And here is a first draft of the unit tests: https://github.com/kellyjonbrazil/salt/blob/jc-renderer/tests/unit/renderers/test_jc.py~~

I would just like to make sure a Renderer is the proper approach and if there are any other pieces missing, like documentation/Changelog before I submit a PR.

The outputter works, but requires a parser parameter passed to it. I didn't know how to get a parameter into the outputter, so I'm using an environment variable:

$ JC_PARSER=uptime salt '*' cmd.run 'uptime' --out=jc

and here's an example of the serializer in an sls file:

{% set date_out = salt.cmd.run_stdout('date') %}
{% set date_jc = salt.slsutil.deserialize('jc', date_out, parser='date') %}

test:
    cmd.run:
        - name: >
            echo 'The timezone is {{ date_jc.timezone }}'

The only issue with this setup is that jc needs to be installed on the minions for this to work. I was hoping to have the deserialization happen on the master. If anyone has any suggestions on that I'm open.

Thanks, Kelly

kellyjonbrazil commented 2 years ago

Note, there is a new high-level API for calling jc parsers that can simplify the implementation a bit:

https://github.com/kellyjonbrazil/jc/tree/master/docs

Also, there are now a handful of 'streaming' parsers that return generators that can be iterated over. Not sure if there is a use case for those. In any case, for every streaming parser there is a corresponding 'standard' parser, so streaming parsers can be filtered out if they are not useful.