YDK fails to decode RPC received from Juniper device

Expected Behavior

YDK is expected to work across wide range of devices, which implement standard management protocols like Netconf, Restconf, and gNMI

Current Behavior

YDK fails to decode RPC received from Juniper device due to failure to correctly extract namespaces from RPC data. Error message:

2019-12-11 12:10:33,682 - ydk - DEBUG - Extracting module namespaces from XML payload
namespace error : Namespace prefix junos for commit-seconds on configuration is not defined
65281" junos:commit-localtime="2019-12-04 13:14:41 UTC" junos:commit-user="root"

As a result YDK fails to import YANG modules from device:

2019-12-11 12:10:33,683 - ydk - DEBUG - Trace: Writing message (session 63557): 
<?xml version="1.0"?>
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="2">
  <get-schema xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
    <identifier>//xml.juniper.net/xnm/1.1/xnm</identifier>
    <format>yang</format>
  </get-schema>
</rpc>

2019-12-11 12:10:33,683 - ydk - DEBUG - NetconfSSHClient: NC session status: 1
2019-12-11 12:10:33,683 - ydk - DEBUG - Netconf SSH Client: receiving reply RPC
2019-12-11 12:10:33,699 - ydk - DEBUG - Trace: Received message (session 63557): 
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/19.2R0/junos" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="2">
<rpc-error>
<error-type>protocol</error-type>
<error-tag>operation-failed</error-tag>
<error-severity>error</error-severity>
<error-message>
invalid schema identifier : //xml.juniper.net/xnm/1.1/xnm
</error-message>
</rpc-error>
</rpc-reply>

with following exception:

ydk.errors.YCoreError:  YCodecError:Data model "//xml.juniper.net/xnm/1.1/xnm" not found.. Path:

Steps to Reproduce

import logging

from ydk.services import NetconfService, Datastore, CodecService
from ydk.providers import NetconfServiceProvider, CodecServiceProvider

logger = logging.getLogger('ydk')
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter(("%(asctime)s - %(name)s - "
                               "%(levelname)s - %(message)s"))
handler.setFormatter(formatter)
logger.addHandler(handler)

DEVICES = {
    'juno': {
        'ip': '10.10.30.4',
        'pass': 'Juniper',
        'uname': 'admin',
        'port': 830
    },
    'xe': {
        'ip': 'ios-xe-mgmt.cisco.com',
        'pass': 'C1sco12345',
        'uname': 'developer',
        'port': 10000
    }
}

if __name__ == '__main__':
    device = DEVICES['juno']
    provider = NetconfServiceProvider(
        address=device['ip'],
        port=device['port'],
        username=device['uname'],
        password=device['pass'],
        protocol='ssh'
    )

    netconf = NetconfService()
    config = netconf.get_config(provider, Datastore.running)

    codec = CodecService()
    pr = CodecServiceProvider(type='xml')
    xml_list = codec.encode(pr, config, pretty=True)
    print("\nGOT DEVICE CONFIGURATION:\n%s" % '\n'.join(xml_list))

Logs

JUNOS_DEBUG.txt

System Information

YDK-0.8.4

This is not a YDK issue, but an issue with how Junos responds to the RPCs. By default junos uses a single namespace, non rfc/yang compatible. To change this behaviour configure the junos device with:

set system services netconf rfc-compliant
set system services netconf yang-compliant

rfc-compliant will enable xml namespaces to match the yang models namespaces
yang-compliant will results in:
- suppressing <junos:comment> and <undocumented> elements
- YANG compliant serialization of lists with choice-ident/choice-value keys (there are a few yang lists with multiple leaf keys that have a different way of serialization in the native junos compared to rfc/yang.

On top of this you may be hitting a bug that older junos versions had with get-schema (https://github.com/ncclient/ncclient/issues/276). The newest ncclient has a workaround (). However, the workaround applies for the "juniper" device flavour, which returns NCElement objects from the rpc response instead of RPCReply objects like vanilla netconf devices (https://github.com/ncclient/ncclient/issues/453), so ydk probably won't understand this non-vanilla reply object.

A recent junos version (which has the fix for get-schema) configured with rfc-compliant and yang-compliant should not face this problem.

CiscoDevNet / ydk-gen