Support for unique leaf (or leaf combo) in a list of lists

[Reshad-Rahman]

If I have a list which has a container, I can specify a leaf to be unique or a combination of leaf nodes to be unique. e.g.:

    list a-list-of-containers {
      key "name";
      unique "attr1 attr2";
      unique "cont1/attr1 cont1/attr2";
      unique "attr1 cont1/attr2";
      leaf name {
        type string;
      }
      leaf attr1 {
        type int32;
      }
      leaf attr2 {
        type string;
      }
      container cont1 {
        leaf attr1 {
          type int32;
        }
        leaf attr2 {
          type string;
        }
      }
    }

If I have a list of lists, I know of no way to have a leaf inside the lower list to be delcared unique across all lists. In the example below, let's say I want to define the leaf sin (social insurance number) to be unique across all provinces.

list a-list-of-lists {
  key "province";

  leaf province {
    type string;
  }
  list employees {
    key "sin";

    leaf sin {
      type int32;
    }
  }
}

I tried with leafref but failed, however maybe I didn't try hard enough.

[llhotka]

This has already been discussed a few times, e.g. in this thread.

I don't know remember the resolution was (if there was any) but it seems that nothing in sec. 7.8.3 of RFC 7950 prevents this.

[mbj4668]

Yes it works for ensuring that a single leaf is unique within a nested sublist. But it doesn't work for the case where the combination of two leafs are supposed to be unique, e.g., suppose you want the combination of (ip, port) to be unique in all /a/b/servers:

     container a {
       list b {
         ...
         list server {
           ...
           leaf ip { ... }
           leaf port { ... }
         }
       }
     }

We have solved this like XSD does, with two extensions:

     container a {
       tailf:unique-selector 'b/server' {
         tailf:unique-leaf 'ip';
         tailf:unique-leaf 'port';
       }
       list b {
         ...
         list server {
           ...
           leaf ip { ... }
           leaf port { ... }
         }
       }
     }

It means that "foreach b/server, the combination of (ip,port) must be unqiue"

[llhotka]

@mbj4668 I think it should work out of the box.

[mbj4668]

You mean "unique "b/server/ip b/server/port"?

I don't see how that could work. The current text says:

the combined values of all the leaf instances [...] MUST be unique

so if we have: b 1 server 1 ip 10.0.0.1 port 22 b 1 server 2 ip 10.0.0.1 port 23 b 2 server 3 ip 10.0.0.1 port 23 b 2 server 4 ip 10.0.0.1 port 25

what is the combined value of all leaf instances?

[llhotka]

It could work as defined in sec. 12.16 of RFC 6110.

[Reshad-Rahman]

It could work as defined in sec. 12.16 of RFC 6110.

Lada, so all we'd need is to allow this in next revision of YANG?

[mbj4668]

No, this doesn't solve the problem! The algorithm in 6110 works as long as there are no nested lists.

When you have a nested list, if you were to apply the algorithm in 6110 you would not get the result you expect. We want the combination of (ip,port) to be unique, but the alg in 6110 will not guarantee this.

So if we think that this is a use case we want to support, new statements are needed.

[llhotka]

@mbj4668 This is not true. It would work as expected for nested lists in Reshad's example a-list-of-lists because there is only one leaf that is required to be unique.

With two or more leaves, it would also work in the sense that two entries of the top-level list cannot contain the same combination of leaf instances described by the schema node identifiers in unique. Of course, as unique addresses the enclosing (outer) list, it cannot take into account any further aggregation inside the entries of the outer list. A must expression should be used in such cases.

[mbj4668]

I tried this with two leafs. The intention is that no server can the same (ip,port):

module a {
  namespace urn:a;
  prefix a;

  container a {
    list b {
      key id;
      leaf id { type string; }
      unique "server/ip server/port";
      list server {
        key id;
        leaf id { type string; }
        leaf ip { type string; }
        leaf port { type string; }
      }
    }
  }
}

and data:

<nc:data xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
         xmlns="urn:a">
  <a>
    <b>
      <id>1</id>
      <server>
        <id>1</id>
        <ip>10.0.0.1</ip>
        <port>22</port>
      </server>
      <server>
        <id>2</id>
        <ip>10.0.0.2</ip>
        <port>23</port>
      </server>
    </b>
    <b>
      <id>2</id>
      <server>
        <id>3</id>
        <ip>10.0.0.1</ip>
        <port>23</port>
      </server>
    </b>
  </a>
</nc:data>

The rules in 6110 gives:

Violated uniqueness for "a:server/a:ip a:server/a:port"

Which is clearly not the intention.

So I don't think we can say that this "just works". I agree that we can say that we don't want to support this. I think the original proposal is to add direct support for this use case.

[llhotka]

Intentions can be different but I do argue that the 6110 algorithm does exactly what the specification in RFC 7950 says:

   The "unique" constraint specifies that the combined values of all the
   leaf instances specified in the argument string, including leafs with
   default values, MUST be unique within all list entry instances in
   which all referenced leafs exist or have default values.

This is clearly violated in your XML instance data: each of the b entries contains the combination server/ip = 10.0.0.1 and server/port = 23. If the intention is that the combination is conflicting only if it is contained in the same entry of the inner list, then it is just an extra requirement that the unique statement cannot take into account.

[mbj4668]

Right. So this issue is about adding support for this.

[llhotka]

IMO this rather rare, and specific conditions in a particular case may be different, e.g list inside list inside list. A must contraint is a better and more general solution. Even if the corresponding expression may not be very intuitive, an explaining description should suffice.

[Reshad-Rahman]

I don't think this is that rare. You just need 2 lists (list inside list), here's the error I get from pyang 1.7.5 with 1 leaf. And uniqueness of combination of 2 leaf nodes is not uncommon.

error: the identifier "employees" in the unique argument references a list; this is not legal

list a-list-of-lists {
  key "province";
  unique "employees/sin";

  leaf province {
    type string;
  }
  list employees {
    key "sin";

    leaf sin {
      type int32;
    }
  }
}

[llhotka]

@Reshad-Rahman, as I wrote, the DSDL algorithm would work just fine with your example. Pyang parser is IMO hyper-correct in this case.

If the DSDL algorithm doesn't do what the data modeller wants, then I would suggest to use a must statement rather than introduce a new construct in YANG.

[kwatsen]

+1 vote for wanting to see a solution for this.

Sent to yang-doctors's list, which quickly pointed here. The YD thread is here.

 Assume data model:

  +—rw tenants
      +—rw tenant [key]
          +—rw  key        string
          +—rw admins
              +—rw admin [name]
                   +—rw name     string

 The "admin" list’s key “name” ensures that the names are unique per tenant, but not globally.   Is it possible to place a “must” statement on the “tenants” container to ensure globally-unique admin names?

 Can anyone offer syntax to do so?

[Reshad-Rahman]

Update from YD list:

Sorry, it should be

must "not(preceding-sibling::tenant[admins/admin/name = current()/admins/admin/name])

I thought "preceding-sibling” was for selecting a sibling? In my example, “tenant” doesn’t have a sibling, so what is selected? You are putting the must statement on the “tenant” node, right?

Yes. In your example, "tenant" is a list, and its instances/entries are sibling nodes in the XPath data model. So the rule states that there must not be two "tenant" entries containing admins/admin/leaf with the same value. If this condition is violated, it is reported for the second "tenant" entry.

The same rule can be written in other ways, too, but the advantage of preceding-sibling:: is that the check is performed only "above the diagonal" of all pairwise combinations of "tenant" entries.

Lada

[abierman]

really long discussion but not convinced this additional complexity will be seen as an improvement. The unique-stmt is rarely used compared to must-stmt.

Do not add: complexity high, bc: low, importance: low

[kwatsen]

Regarding importance, this use-case seems to be one that Reshad, Andy, myself independently wrote messages to lists about.

must expressions are powerful, yet not very readable, and one has to be an XPath expert, which is why I assume tail-f created special syntax to support this use-case.

[janlindblad]

Even if we have a proprietary solution for this, I don't think this is very high prio for YANG 2.0

[kwatsen]

balazs: must statements are slow andy: how important is this? fallback (must) express exist rob: agree w/ andy

[BalazsLengyel]

this works according to rfc7950. I would like this clarified whether it should work.