search

CiscoDevNet / ydk-gen

Generate model-driven APIs from YANG models
http://ciscodevnet.github.io/ydk-gen/
Apache License 2.0
137 stars 74 forks source link

CodecServiceProvider fails due to Libyang error #827

Closed ygorelik closed 2 years ago

ygorelik commented 6 years ago

Issue tracker is ONLY used for reporting bugs. Please use the YDK Community for any support issues.

Expected Behavior

Expected the CodecService convert Cisco-IOS-XR-ifmgr-cfg:interface-configurations/interface-configuration entities to JSON string.

Current Behavior

The conversion fails with error:

2018-10-11 08:53:44,861 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Invalid keyword "type" as a child to "annotation". Path: '/'
2018-10-11 08:53:44,861 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Module "yang" parsing failed.
2018-10-11 08:53:44,861 - ydk - ERROR - Could not create repository in: /usr/local/lib/python2.7/dist-packages/ydk/models/cisco_ios_xr/_yang

Similar issue is described in the Libyang github.

Your Script

from ydk.services import CRUDService
from ydk.path import Repository, Codec
from ydk.gnmi.providers import gNMIServiceProvider

from ydk.models.cisco_ios_xr import Cisco_IOS_XR_ifmgr_cfg as ifmgr
from ydk.types import Empty

def run_test(provider):
    root = provider.get_session().get_root_schema()
    crud = CRUDService()

    """Read interface configuration"""
    ifc_filter = ifmgr.InterfaceConfigurations()
    ifc = ifmgr.InterfaceConfigurations.InterfaceConfiguration()
    ifc.active = 'act'
    ifc.interface_name = "Loopback10"
    ifc_filter.interface_configuration.append(ifc)

    ifc_read = crud.read(provider, ifc_filter)
    if ifc_read is not None:
        from ydk.services import CodecService
        from ydk.providers import CodecServiceProvider
        codec_service = CodecService()
        codec_provider = CodecServiceProvider(type='json')
        payload = codec_service.encode(codec_provider, ifc_read)
        print('CREATED INTERFACE CONFIGURATION:')
        print(payload)

if __name__ == "__main__":

    """Execute main program."""
    parser = ArgumentParser()
    parser.add_argument("-v", "--verbose", help="print debugging messages",
                        action="store_true")
    parser.add_argument("device",
                        help="NETCONF device (ssh://user:password@host:port)")
    args = parser.parse_args()
    device = urlparse(args.device)

    # log debug messages if verbose argument specified
    if args.verbose:
        import logging
        enable_logging(logging.DEBUG)

    # create gNMI service provider
    repo = Repository("/home/ygorelik/ydk-gen/scripts/samples/repository/192.168.122.107")
    provider = gNMIServiceProvider(repo=repo, address=device.hostname,
                                      port=device.port,
                                      username=device.username,
                                      password=device.password)
    run_test(provider)

Logs

Enable logging and post the logs below

2018-10-11 08:53:44,860 - ydk - DEBUG - Creating libyang context in path: /usr/local/lib/python2.7/dist-packages/ydk/models/cisco_ios_xr/_yang
2018-10-11 08:53:44,860 - ydk - DEBUG - [libyang] Extension plugin "/usr/local/lib/libyang/metadata.so" already loaded.
2018-10-11 08:53:44,861 - ydk - DEBUG - [libyang] Extension plugin "/usr/local/lib/libyang/libyang_ext_test.so" already loaded.
2018-10-11 08:53:44,861 - ydk - DEBUG - [libyang] Extension plugin "/usr/local/lib/libyang/nacm.so" already loaded.
2018-10-11 08:53:44,861 - ydk - DEBUG - [libyang] Reading module "ietf-yang-metadata".
2018-10-11 08:53:44,861 - ydk - DEBUG - [libyang] Module "ietf-yang-metadata@2016-08-05" successfully parsed as implemented.
2018-10-11 08:53:44,861 - ydk - DEBUG - [libyang] Reading module "yang".
2018-10-11 08:53:44,861 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Invalid keyword "type" as a child to "annotation". Path: '/'
2018-10-11 08:53:44,861 - ydk - ERROR - Data is invalid according to the yang model. Libyang error: Module "yang" parsing failed.
2018-10-11 08:53:44,861 - ydk - ERROR - Could not create repository in: /usr/local/lib/python2.7/dist-packages/ydk/models/cisco_ios_xr/_yang
Traceback (most recent call last):
  File "xr-ifmgr-int.py", line 115, in <module>
    run_test(provider)
  File "xr-ifmgr-int.py", line 81, in run_test
    payload = codec_service.encode(codec_provider, ifc_read)
  File "/usr/local/lib/python2.7/dist-packages/ydk/errors/error_handler.py", line 112, in helper
    return func(self, provider, entity, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/ydk/services/codec_service.py", line 78, in encode
    return self._encode(provider, entity_holder, pretty, subtree)
  File "/usr/local/lib/python2.7/dist-packages/ydk/services/codec_service.py", line 96, in _encode
    provider.initialize(bundle_name, _get_yang_path(entity))
  File "/usr/local/lib/python2.7/dist-packages/ydk/providers/codec_provider.py", line 81, in initialize
    self._initialize_root_schema(bundle_name, repo)
  File "/usr/local/lib/python2.7/dist-packages/ydk/providers/codec_provider.py", line 112, in _initialize_root_schema
    self._root_schema_table[name] = repo.create_root_schema(lookup_tables, capabilities)
RuntimeError: YIllegalStateError: Could not create repository in: /usr/local/lib/python2.7/dist-packages/ydk/models/cisco_ios_xr/_yang
2018-10-11 08:53:44,875 - ydk - INFO - Disconnected from device

System Information

YDK0.7.3 + cisco_ios_xr (6.5.1)

ygorelik commented 5 years ago

This issue appears only in Python. As a workaround the XmlSubtreeCodec or JsonSubtreeCodec can be used. Here is an example:

    ifc_read = crud.read(provider, ifc_filter)
    if ifc_read is not None:
        from ydk.entity_utils import JsonSubtreeCodec
        jcodec = JsonSubtreeCodec()
        payload = jcodec.encode(ifc_read, provider.get_session().get_root_schema(), True)
        print('CREATED INTERFACE CONFIGURATION:')
        print(payload)

The resulting output:

2019-08-20 16:41:15,635 - ydk - DEBUG - JsonCodec: Children count for 'regular-address[address='2001:db8::ff:101']': 0
2019-08-20 16:41:15,636 - ydk - INFO - Disconnected from device
CREATED INTERFACE CONFIGURATION:
{
  "Cisco-IOS-XR-ifmgr-cfg:interface-configurations": {
    "interface-configuration": [
      {
        "Cisco-IOS-XR-ipv4-io-cfg:ipv4-network": {
          "addresses": {
            "primary": {
              "address": "172.16.255.1",
              "netmask": "255.255.255.255"
            }
          }
        },
        "Cisco-IOS-XR-ipv6-ma-cfg:ipv6-network": {
          "addresses": {
            "regular-addresses": {
              "regular-address": [
                {
                  "address": "2001:db8::ff:1",
                  "prefix-length": 128,
                  "zone": 0
                },
                {
                  "address": "2001:db8::ff:101",
                  "prefix-length": 128,
                  "zone": 0
                }
              ]
            }
          }
        },
        "active": "act",
        "description": "PRIMARY ROUTER LOOPBACK",
        "interface-name": "Loopback0",
        "interface-virtual": null
      }
    ]
  }
}
ygorelik commented 2 years ago

Not reproducible with YDK-0.8.6.

Python script based on CiscoDevNet sandbox

from ydk.services import CRUDService
from ydk.path import Repository
from ydk.gnmi.providers import gNMIServiceProvider

from ydk.models.test_cisco_ios_xr import Cisco_IOS_XR_ifmgr_cfg as ifmgr

from argparse import ArgumentParser
from urllib.parse import urlparse

def enable_logging(level):
    log = logging.getLogger('ydk')
    log.setLevel(level)
    handler = logging.StreamHandler()
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    handler.setFormatter(formatter)
    log.addHandler(handler)

def run_test(service_provider):
    crud = CRUDService()

    """Read interface configuration"""
    ifc_filter = ifmgr.InterfaceConfigurations()
    ifc = ifmgr.InterfaceConfigurations.InterfaceConfiguration()
    ifc.active = '"act"'
    ifc.interface_name = '"Loopback30"'
    ifc_filter.interface_configuration.append(ifc)

    ifc_read = crud.read(service_provider, ifc_filter)
    if ifc_read is not None:
        from ydk.services import CodecService
        from ydk.providers import CodecServiceProvider
        codec_service = CodecService()
        codec_provider = CodecServiceProvider(type='json')
        payload = codec_service.encode(codec_provider, ifc_read)
        print('INTERFACE CONFIGURATION:')
        print(payload)

if __name__ == "__main__":

    """Execute main program."""
    parser = ArgumentParser()
    parser.add_argument("-v", "--verbose", help="print debugging messages",
                        action="store_true")
    parser.add_argument("device",
                        help="NETCONF device (ssh://user:password@host:port)")
    args = parser.parse_args()
    device = urlparse(args.device)

    # log debug messages if verbose argument specified
    if args.verbose:
        import logging
        enable_logging(logging.INFO)

    # create gNMI service provider
    repo = Repository("/Users/ygorelik/gitlab/ydk-gen/scripts/dev/yang/cisco-ios-xr")

    provider = gNMIServiceProvider(repo=repo, address=device.hostname,
                                   port=device.port,
                                   username=device.username,
                                   password=device.password,
                                   server_certificate="/Users/ygorelik/gitlab/ydk-gen/scripts/certificates/sandbox-iosxr-1.cisco.com.pem",
                                   )
    run_test(provider)

Script output

/Users/ygorelik/venv3.7/bin/python3.7 /Users/ygorelik/gitlab/ydk-gen/scripts/tests/test_827.py -v ssh://admin:C1sco12345@sandbox-iosxr-1.cisco.com:57777
2021-11-30 11:26:27,035 - ydk - INFO - gNMIServiceProvider Connected to sandbox-iosxr-1.cisco.com via Secure Channel
2021-11-30 11:26:27,039 - ydk - INFO - Executing CRUD read operation on [Cisco-IOS-XR-ifmgr-cfg:interface-configurations]
2021-11-30 11:26:27,045 - ydk - INFO - 
=============== Get Request Sent ================
path {
  origin: "Cisco-IOS-XR-ifmgr-cfg"
  elem {
    name: "interface-configurations"
  }
  elem {
    name: "interface-configuration"
    key {
      key: "active"
      value: "\"act\""
    }
    key {
      key: "interface-name"
      value: "\"Loopback30\""
    }
  }
}
encoding: JSON_IETF

2021-11-30 11:26:27,097 - ydk - INFO - 
============= Get Response Received =============
notification {
  timestamp: 1638300345798427223
  update {
    path {
      origin: "Cisco-IOS-XR-ifmgr-cfg"
      elem {
        name: "interface-configurations"
      }
      elem {
        name: "interface-configuration"
        key {
          key: "active"
          value: "\"act\""
        }
        key {
          key: "interface-name"
          value: "\"Loopback30\""
        }
      }
    }
    val {
      json_ietf_val: "[{\"active\":\"act\",\"interface-name\":\"Loopback30\",\"interface-virtual\":[null],\"description\":\"NETCONF Configured\",\"Cisco-IOS-XR-ipv4-io-cfg:ipv4-network\":{\"addresses\":{\"primary\":{\"address\":\"7.7.7.7\",\"netmask\":\"255.255.255.255\"}}}}]"
    }
  }
}
error {
}

2021-11-30 11:26:27,098 - ydk - INFO - Get Operation Succeeded
INTERFACE CONFIGURATION:
{
  "Cisco-IOS-XR-ifmgr-cfg:interface-configurations": {
    "interface-configuration": [
      {
        "active": "act",
        "interface-name": "Loopback30",
        "interface-virtual": null,
        "description": "NETCONF Configured",
        "Cisco-IOS-XR-ipv4-io-cfg:ipv4-network": {
          "addresses": {
            "primary": {
              "address": "7.7.7.7",
              "netmask": "255.255.255.255"
            }
          }
        }
      }
    ]
  }
}

2021-11-30 11:26:27,152 - ydk - INFO - Disconnected from device

Process finished with exit code 0