search

clicon / clixon

YANG-based toolchain including NETCONF and RESTCONF interfaces and an interactive CLI
http://www.clicon.org/
Other
215 stars 72 forks source link

Cannot get child using XPATH #414

Open dima1308 opened 1 year ago

dima1308 commented 1 year ago

When I execute the following query:

<get>
  <filter type="subtree">
    <rb-tg-if:interfaces xmlns:rb-tg-if="http://siklu.com/yang/tg/interfaces"/>
  </filter>
</get>

I get the following response:

<?xml version="1.0" encoding="utf-8"?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <interfaces xmlns="http://siklu.com/yang/tg/interfaces">
    <host>
      <state>
        <mac-address>00:24:a4:16:51:a1</mac-address>
        <counters>
          <in-octets>157432</in-octets>
          <in-pkts>3730</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>4</in-no-rule-discards>
          <out-octets>6791986</out-octets>
          <out-pkts>60877</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </host>
    <ports>
      <name>eth1</name>
      <alias>myeth1</alias>
      <admin-status>up</admin-status>
      <copper-specific-config>
        <auto-negotiate>true</auto-negotiate>
      </copper-specific-config>
      <connector-type>RJ-45</connector-type>
      <port-speed>10Gbps</port-speed>
      <state>
        <oper-status>up</oper-status>
        <actual-duplex-mode>full</actual-duplex-mode>
        <actual-port-speed>1Gbps</actual-port-speed>
        <counters>
          <in-octets>6407339</in-octets>
          <in-pkts>55797</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>5</in-no-rule-discards>
          <out-octets>532508</out-octets>
          <out-pkts>8699</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </ports>
    <ports>
      <name>eth2</name>
      <alias>myeth2</alias>
      <admin-status>up</admin-status>
      <copper-specific-config>
        <auto-negotiate>true</auto-negotiate>
        <pse-out>true</pse-out>
      </copper-specific-config>
      <connector-type>RJ-45</connector-type>
      <port-speed>1Gbps</port-speed>
      <state>
        <oper-status>down</oper-status>
        <counters>
          <in-octets>0</in-octets>
          <in-pkts>0</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>0</in-no-rule-discards>
          <out-octets>0</out-octets>
          <out-pkts>0</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </ports>
    <ports>
      <name>eth3</name>
      <alias>myeth3</alias>
      <admin-status>up</admin-status>
      <copper-specific-config>
        <auto-negotiate>true</auto-negotiate>
      </copper-specific-config>
      <connector-type>SFP+</connector-type>
      <port-speed>10Gbps</port-speed>
      <state>
        <oper-status>down</oper-status>
        <counters>
          <in-octets>0</in-octets>
          <in-pkts>0</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>0</in-no-rule-discards>
          <out-octets>0</out-octets>
          <out-pkts>0</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </ports>
    <rf-interface>
      <name>rf-23654789</name>
      <state>
        <oper-status>up</oper-status>
        <counters>
          <in-octets>22536898</in-octets>
          <in-pkts>33518</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>0</in-no-rule-discards>
          <out-octets>4706450</out-octets>
          <out-pkts>31432</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </rf-interface>
    <tunnels>
      <name>tunnel-1</name>
      <oper-status>unknown</oper-status>
      <counters>
        <in-octets>0</in-octets>
        <in-pkts>0</in-pkts>
        <in-discards>0</in-discards>
        <in-errors>0</in-errors>
        <in-no-rule-discards>0</in-no-rule-discards>
        <out-octets>0</out-octets>
        <out-pkts>0</out-pkts>
        <out-discards>0</out-discards>
        <out-errors>0</out-errors>
      </counters>
    </tunnels>
  </interfaces>
</data>

Now, I'm only trying to get a list of interface names using the following query:

<get>
  <filter xmlns:rb-tg-if="http://siklu.com/yang/tg/interfaces" type="xpath"
          select="/rb-tg-if:interfaces/rb-tg-if:ports/rb-tg-if:name"/>
</get>

I got the expected result:

<?xml version="1.0" encoding="utf-8"?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <interfaces xmlns="http://siklu.com/yang/tg/interfaces">
    <ports>
      <name>eth1</name>
    </ports>
    <ports>
      <name>eth2</name>
    </ports>
    <ports>
      <name>eth3</name>
    </ports>
  </interfaces>
</data>

Next, I am trying to get interfaces with alias='myeth2':

<get>
  <filter xmlns:rb-tg-if="http://siklu.com/yang/tg/interfaces" type="xpath"
          select="/rb-tg-if:interfaces/rb-tg-if:ports[rb-tg-if:alias='myeth2']"/>
</get>

I received the following result, as expected (we have one interface matching the condition):

<?xml version="1.0" encoding="utf-8"?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <interfaces xmlns="http://siklu.com/yang/tg/interfaces">
    <ports>
      <name>eth2</name>
      <alias>myeth2</alias>
      <admin-status>up</admin-status>
      <copper-specific-config>
        <auto-negotiate>true</auto-negotiate>
        <pse-out>true</pse-out>
      </copper-specific-config>
      <connector-type>RJ-45</connector-type>
      <port-speed>1Gbps</port-speed>
      <state>
        <oper-status>down</oper-status>
        <counters>
          <in-octets>0</in-octets>
          <in-pkts>0</in-pkts>
          <in-discards>0</in-discards>
          <in-errors>0</in-errors>
          <in-no-rule-discards>0</in-no-rule-discards>
          <out-octets>0</out-octets>
          <out-pkts>0</out-pkts>
          <out-discards>0</out-discards>
          <out-errors>0</out-errors>
        </counters>
      </state>
    </ports>
  </interfaces>
</data>

And now, I'm trying to get only the name of the interface matching the same condition as above:

<get>
  <filter xmlns:rb-tg-if="http://siklu.com/yang/tg/interfaces" type="xpath"
          select="/rb-tg-if:interfaces/rb-tg-if:ports[rb-tg-if:alias='myeth2']/rb-tg-if:name"/>
</get>

I got an empty response - which is not expected since the previous query returns one record matching the condition:

<?xml version="1.0" encoding="utf-8"?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"/>

if I change the query as shown below, I got both values: name and alias in the response: The request:

<get>
  <filter xmlns:rb-tg-if="http://siklu.com/yang/tg/interfaces" type="xpath"
          select="/rb-tg-if:interfaces/rb-tg-if:ports[rb-tg-if:alias='myeth2']/rb-tg-if:alias"/>
</get>

The response:

<?xml version="1.0" encoding="utf-8"?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <interfaces xmlns="http://siklu.com/yang/tg/interfaces">
    <ports>
      <name>eth2</name>
      <alias>myeth2</alias>
    </ports>
  </interfaces>
</data>

In this case, I expect to get output that will only include alias

We use CLIXON from Jan 20, 2023, hash: b3dcee96393a261984e64b5790ea1725b5d1f6ce

To summarize, I saw two issues:

  1. In some cases, an empty response for the request
  2. Not expected value in the response.

Please see attached yang models we use: yang.zip

olofhagsand commented 1 year ago

I have recreated the problem. The XPATH function works, but is broken with the NETCONF filter handling.

olofhagsand commented 1 year ago

The two cases above:

  1. In some cases, an empty response for the request
  2. Not expected value in the response.

The two cases are quite different, where (1) is in fact a bug, while (2) follows RFC6241

Issue (1) is a larger issue that requires some restructuring of the xmldb code + default values and is not an easy fix. It is a necessary refactoring that has been planned for some time. In short, it has to do with how the function works:

  1. Get the requested config without default values and apply xpath.
  2. Add default values according to with-defaults setting
  3. Add state data
  4. Make a new xpath (to filter default and state data). The problem is that xpath is made in two steps (1 and 4). Information relevant to the xpath may therefore be removed in step 1 and is not relevant in step 4. This explains issue : "In some cases, an empty response for the request". This is a bug.

Issue (2), RFC 6241 Section 8.9.1 says:

The response message contains the subtrees selected by the filter expression. For each such subtree, the path from the data model root node down to the subtree, including any elements or attributes necessary to uniquely identify the subtree, are included in the response message. Specific data instances are not duplicated in the response.

which clixon interprets as YANG keys must be included "to identify the subtree". A client could make an xpath call (ie clixon_snmp) after the the call to make a proper XPATH filtering where the key nodes will be removed. Therefore this is not considered a bug.

dima1308 commented 1 year ago

@olofhagsand may I ask you what is the status of the fix for this?

olofhagsand commented 1 year ago

The refactoring necessary for this was initially planned for 6.2, but has now been pushed to 6.3. Refactoring of netconf input code was made before.

dima1308 commented 1 year ago

@olofhagsand Can I inquire about the status of the fix? It was scheduled for 6.3, but from what I can observe, it was not incorporated into the release.

olofhagsand commented 1 year ago

Hm, it says it is fixed, and there is a patch, changelog entry and testcases. But it is not closed, which it should be (I will close it). Do you see it not fixed in some way? --Olof

On 2023-07-30 17:44, Dmitry Grinberg wrote:

@olofhagsand https://github.com/olofhagsand Can I inquire about the status of the fix? It was scheduled for 6.3, but from what I can observe, it was not incorporated into the release.

— Reply to this email directly, view it on GitHub https://github.com/clicon/clixon/issues/414#issuecomment-1657204822, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGD65IS2CPOUPXU4GDK3HDXSZ6NHANCNFSM6AAAAAAUMZHE7A. You are receiving this because you were mentioned.Message ID: @.***>

dima1308 commented 1 year ago

This bug prevents us from merging https://github.com/clicon/clixon/pull/415. I guess, we can merge it now since this bug is fixed. Correct?

shmuelhazan commented 1 year ago

@dima1308 @olofhagsand Looks like this bug has not been fixed (https://github.com/clicon/clixon/pull/415 fails still, due to this bug)

olofhagsand commented 1 year ago

@dima1308 @shmuelhazan . On closer inspection, this refers to a refactoring that has not been made yet. Thus issue (1) above still remains and is not fixed.

dima1308 commented 3 weeks ago

@olofhagsand any plans to fix it?

olofhagsand commented 3 weeks ago

Need to revisit. This code has been refactored.