Support performant load_config_file(...) for TEXT format

[pheller]

For users most comfortable with the curly-brace / junos style configuration (FORMAT_TEXT or simply "TEXT" format in Clixon terms), it is desirable to support round-trip of such configuration syntax. That is, one could capture the output of cli_auto_show(...) of a configuration in this format, manipulate it as required, then submit it to load_config_file(...) to either merge or overwrite the running configuration.

Two possible implementation approaches are as follows, and they involve translation to a single XML payload, so as to avoid the penalty of a frontend-backend roundtrip and semantic check for each element of the configuration.

Approaches:

1) Transform the TEXT format to CLI format, then transform the CLI format to an XML payload, then invoke load_config_file(...) with this payload.

2) Transform the TEXT format directly to XML and invoke load_config_file(...) with this payload.

The former option is attractive since code for translation from TEXT to CLI format is available, and if the transform from CLI to XML is implemented, then load_config_file(...) performance will improve for large CLI payloads as well.

[olofhagsand]

the curly-brace / junos style configuration (FORMAT_TEXT or simply "TEXT" format in Clixon terms),

it would be good with a documented specification of this format.

[pheller]

So far, the closest I've found to a specification is here.

[olofhagsand]

Nice example. It would be nice to have corresponding XML for that example, eg protocol ospf and/or policy-option. My follow-up question is whether the text format can be translated to XML directly with no other information? If so it would be straightforward to do. But it may be that the YANG is necessary as well. For example, policy.-option YANG

[olofhagsand]

Experimental development is here: https://github.com/clicon/clixon/tree/textparser

[olofhagsand]

This is now completed:

• pasrsing and loading of TEXT files
• syntax extended for namespace, leaf-lists and lists keys. Please verify and close if no issues found.

[pheller]

I've given the feature a test, and it looks pretty good, however there is a lingering error:

Some defaults omitted in the examples to follow for clarity.

This XML configuration snippet:

   <network-instances xmlns="http://openconfig.net/yang/network-instance">
      <network-instance>
         <name>default</name>
         <config>
            <name>default</name>
            <type>oc-ni-types:DEFAULT_INSTANCE</type>
            <enabled>true</enabled>
            <router-id>1.2.3.4</router-id>
         </config>
         <table-connections>
            <table-connection>
               <src-protocol>oc-pol-types:LOCAL_AGGREGATE</src-protocol>
               <dst-protocol>oc-pol-types:BGP</dst-protocol>
               <address-family>oc-types:IPV4</address-family>
               <config>
                  <src-protocol>oc-pol-types:LOCAL_AGGREGATE</src-protocol>
                  <address-family>oc-types:IPV4</address-family>
                  <dst-protocol>oc-pol-types:BGP</dst-protocol>
                  <disable-metric-propagation>false</disable-metric-propagation>
                  <import-policy>TAG_ORIGINATED_PREFIXES</import-policy>
                  <default-import-policy>REJECT_ROUTE</default-import-policy>
               </config>
            </table-connection>

yields this TEXT format snippet

    openconfig-network-instance:network-instances {
        network-instance default {
            config {
                name default;
                type oc-ni-types:DEFAULT_INSTANCE;
                enabled true;
                router-id 1.2.3.4;
            }
            table-connections {
                table-connection oc-pol-types:LOCAL_AGGREGATE oc-pol-types:BGP oc-types:IPV4 {
                    config {
                        src-protocol oc-pol-types:LOCAL_AGGREGATE;
                        address-family oc-types:IPV4;
                        dst-protocol oc-pol-types:BGP;
                        disable-metric-propagation false;
                        import-policy [
                            TAG_ORIGINATED_PREFIXES
                             REJECT_ROUTE
                           ]
                    }
                }
...

It seems that the construction of the leaf-list import-policy is consuming the subsequent leaf default-import-policy.

It is worth nothing that we have not explicitly configured default-import-policy REJECT_ROUTE; - rather it appears in the output as it is the schema-defined default value.

So, with Clixon HEAD, all instances of import-policy leaf-list followed by default-import-policy are rendered this way. There is at least one other similar element rendered this way:

<config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <oc-rpol:routing-policy xmlns:oc-rpol="http://openconfig.net/yang/routing-policy">
    <oc-rpol:policy-definitions>
      <oc-rpol:policy-definition>
        <oc-rpol:statements>
          <oc-rpol:statement>
            <oc-rpol:conditions>
              <oc-bgp-pol:bgp-conditions xmlns:oc-bgp-pol="http://openconfig.net/yang/bgp-policy">
                <oc-bgp-pol:config>
                  <oc-bgp-pol:afi-safi-in xmlns:oc-bgp-types="http://openconfig.net/yang/bgp-types">oc-bgp-types:IPV4_UNICAST</oc-bgp-pol:afi-safi-in>
                  <oc-bgp-pol:community-set>foo</oc-bgp-pol:community-set>
...

will render as ...

...
                            openconfig-bgp-policy:bgp-conditions {
                                config {
                                    afi-safi-in [
                                        oc-bgp-types:IPV4_UNICAST
                                         foo
                                       ]
...

[olofhagsand]

identified error and fixed when leaf follows leaf-list. Please verify again

[pheller]

Ok, I've confirmed that error is resolved, but there is a larger problem with my specific configuration payloads that is more central to how the XML is generated and processed, which has been documented here: https://github.com/clicon/clixon/issues/337

I think this implementation satisfies the original request, and I am able to round-trip configuration from XML -> text -> XML with complete fidelity.