search

CiscoDevNet / ydk-gen

Generate model-driven APIs from YANG models
http://ciscodevnet.github.io/ydk-gen/
Apache License 2.0
135 stars 74 forks source link

YDK not working 17.06.04 - Path is invalid: Cisco-IOS-XE-native:native #1083

Closed vaniatoperich closed 1 year ago

vaniatoperich commented 1 year ago

Issue tracker is ONLY used for reporting bugs. Please use the YDK Community for any support issues.

Expected Behavior

I would expect to be able to load Native module and use it to read/write from Cisco device on either 16.12.08 or 17.06.04 versions

Current Behavior

On IOS versions 17.06.04 or 17.10.01 I am not able to load the yang modules, I get the following error:

ydk.errors.YInvalidArgumentError:  Path is invalid: Cisco-IOS-XE-native:native

However, version 16.12.08 works without any issues. Seems that Yang models are placed in /root/.ydk/192.168.1.142. If this folder exists and has yang models from 16.12.08 version, and switch gets upgraded to 17.06.04, YDK works as "old" models are kept and not overwritten. If I however remove the folder and start from scratch, I get error. I tested also with 17.10.01 which exhibits the same issue...

I am generating python bindings using ydk-gen, however this seems to be an issue of specific version of models.

Steps to Reproduce

Your Script

def get_facts():
    """Return facts of the device."""

    dev = conn()
    facts = {
        "vendor": "Cisco",
        "os_version": "",
        "hostname": "",
        "serial_number": "",
        "model": "",
        "interfaces": [],
        "validate_capable": False
    }

    # Get hardware info
    hw_type = Cisco_IOS_XE_device_hardware_oper.HwType().hw_type_chassis
    hw_info_filter = Cisco_IOS_XE_device_hardware_oper.DeviceHardwareData.DeviceHardware.DeviceInventory()
    hw_info_filter.hw_type = hw_type
    hw_info_filter.hw_dev_index = 1

    # Get all possible interfaces
    native_filter = Cisco_IOS_XE_native.Native()
    fastethernet = native_filter.Interface.FastEthernet()
    fastethernet.name = YFilter.read
    gigabitethernet = native_filter.Interface.GigabitEthernet()
    gigabitethernet.name = YFilter.read
    twogigabitethernet = native_filter.Interface.TwoGigabitEthernet()
    twogigabitethernet.name = YFilter.read
    fivegigabitethernet = native_filter.Interface.FiveGigabitEthernet()
    fivegigabitethernet.name = YFilter.read
    tengigabitethernet = native_filter.Interface.TenGigabitEthernet()
    tengigabitethernet.name = YFilter.read
    twentyfivegige = native_filter.Interface.TwentyFiveGigE()
    twentyfivegige.name = YFilter.read
    fortygigabitethernet = native_filter.Interface.FortyGigabitEthernet()
    fortygigabitethernet.name = YFilter.read
    hundredgige = native_filter.Interface.HundredGigE()
    hundredgige.name = YFilter.read

    # Append all filters
    native_filter.interface.fastethernet.append(fastethernet)
    native_filter.interface.gigabitethernet.append(gigabitethernet)
    native_filter.interface.twogigabitethernet.append(twogigabitethernet)
    native_filter.interface.fivegigabitethernet.append(fivegigabitethernet)
    native_filter.interface.tengigabitethernet.append(tengigabitethernet)
    native_filter.interface.twentyfivegige.append(twentyfivegige)
    native_filter.interface.fortygigabitethernet.append(fortygigabitethernet)
    native_filter.interface.hundredgige.append(hundredgige)

    # Get software info
    native_filter.version = YFilter.read
    native_filter.hostname = YFilter.read

    # Join all filters
    read_filter = Filter(hw_info_filter, native_filter)

    # Get the results
    result = crud_service.read(dev, read_filter)

    # Split the results
    hw_info = result[hw_info_filter]
    native_info = result[native_filter]

    native_info_json = codec.encode(json_provider, native_info)

    facts.update(
        {
            "os_version": native_info.version,
            "hostname": native_info.hostname,
            "model": hw_info.hw_description,
            "serial_number": hw_info.serial_number,
            "interfaces": json.loads(native_info_json),
            "validate_capable": ("urn:ietf:params:netconf:capability:validate:1.1" in dev.get_capabilities())
        }
    )

    return facts
This is how I initialise the connection:

def init(opts):
    """
    Open the connection to the Cisco device, login, and bind to the
    Resource class
    """
    log.debug("Opening connection to cisco device")

    conn_args = copy.deepcopy(opts.get("proxy", {}))

    # This is SSH-based proxy, so it is not safe to enable
    # multiprocessing.
    opts["multiprocessing"] = False

    # Proxy minimum init variables
    if "address" not in conn_args:
        log.critical("No 'address' key found in pillar for this proxy.")
        return False
    if "port" not in conn_args:
        conn_args["port"] = 830
    if "username" not in conn_args:
        log.critical("No 'username' key found in pillar for this proxy.")
        return False
    if "password" not in conn_args:
        log.critical("No 'password' key found in pillar for this proxy.")
        return False
    if "protocol" not in conn_args:
        conn_args["protocol"] = "ssh"

    cisco_device["conn_args"] = conn_args
    cisco_device["conn_args"].pop("proxytype")

    try:
        cisco_device["conn"] = NetconfServiceProvider(**conn_args)
    except Exception as ex:
        log.error("%s : not able to initiate connection to the device", ex)
        cisco_device["initialized"] = False
        cisco_device["conn"] = None
        return

    cisco_device["initialized"] = True

Logs

Enable logging and post the logs below

[INFO    ] Path where models are to be downloaded: /root/.ydk/192.168.1.142
[DEBUG   ] Creating libyang context in path: /root/.ydk/192.168.1.142
[ERROR   ] Data is invalid according to the yang model. Libyang error: Invalid keyword "+".
[ERROR   ] Data is invalid according to the yang model. Libyang error: Module "Cisco-IOS-XE-native" parsing failed.
[ERROR   ] Data is invalid according to the yang model. Libyang error: Importing "Cisco-IOS-XE-native" module into "Cisco-IOS-XE-vrrp-deviation" failed.
[ERROR   ] Data is invalid according to the yang model. Libyang error: Module "Cisco-IOS-XE-vrrp-deviation" parsing failed.
[DEBUG   ] Creating root data node with path '/Cisco-IOS-XE-native:native'
[ERROR   ] Data is invalid according to the yang model. Libyang error: Module not found. Path: '/Cisco-IOS-XE-native'
[ERROR   ] Path 'Cisco-IOS-XE-native:native' is invalid
[CRITICAL] Failed to load grains defined in grain file cisco_ydk.cisco_ydk in function <LoadedFunc name='cisco_ydk.cisco_ydk'>, error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/ydk/errors/error_handler.py", line 50, in handle_runtime_error
    yield
  File "/usr/local/lib/python3.8/dist-packages/ydk/services/crud_service.py", line 88, in _crud_read
    read_top_entity = crud_call(provider, top_filters)
RuntimeError: YInvalidArgumentError: Path is invalid: Cisco-IOS-XE-native:native

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/loader/__init__.py", line 943, in grains
    ret = funcs[key](**kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/var/cache/salt/proxy/extmods/1009-cisco/grains/cisco_ydk.py", line 37, in cisco_ydk
    return {"cisco_ydk": proxy["cisco_ydk.get_facts"]()}
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/var/cache/salt/proxy/extmods/1009-cisco/proxy/cisco_ydk.py", line 316, in get_facts
    result = crud_service.read(dev, read_filter)
  File "/usr/local/lib/python3.8/dist-packages/ydk/services/crud_service.py", line 60, in read
    return _crud_read(provider, read_filter, False, self._crud.read)
  File "/usr/local/lib/python3.8/dist-packages/ydk/services/crud_service.py", line 88, in _crud_read
    read_top_entity = crud_call(provider, top_filters)
  File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.8/dist-packages/ydk/errors/error_handler.py", line 69, in handle_runtime_error
    raise _exc
ydk.errors.YInvalidArgumentError:  Path is invalid: Cisco-IOS-XE-native:native

System Information

python 3.8.10 os: Ubuntu 20.04 x86_64

Additional info

pyang produces following warning however bundle is generated successfully 

Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:606): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1582): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1585): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1597): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1744): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1747): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:1921): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:2243): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:3945): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:4032): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:4156): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:4450): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5557): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5560): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5573): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5752): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5755): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:5942): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2526 (at Cisco-IOS-XE-interfaces.yang:6311): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-interfaces"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1603): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1813): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1816): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1819): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1822): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1825): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1828): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1831): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1834): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1837): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1840): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1843): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1846): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1849): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1852): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1855): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1858): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:1861): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:3167): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:3223): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2477 (at Cisco-IOS-XE-ip.yang:3725): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ip"
Cisco-IOS-XE-native.yang:2499 (at Cisco-IOS-XE-ipv6.yang:447): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ipv6"
Cisco-IOS-XE-native.yang:2499 (at Cisco-IOS-XE-ipv6.yang:471): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-ipv6"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:230): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:239): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:668): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:723): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:855): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:917): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:1115): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:1130): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:1163): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
Cisco-IOS-XE-native.yang:2796 (at Cisco-IOS-XE-logging.yang:1178): warning: node "Cisco-IOS-XE-native::native" is not found in module "Cisco-IOS-XE-logging"
ygorelik commented 1 year ago

It is definitely not YDK issue. The failure appears in libyang library when processing YANG statement 'enum +;'. Based on YANG-1.1 RFC-7950:

   The "enum" statement, which is a substatement to the "type"
   statement, MUST be present if the type is "enumeration".  It is
   repeatedly used to specify each assigned name of an enumeration type.
   It takes as an argument a string that is the assigned name.

the enum argument must be a string and libyang expects that the string value is enclosed in double quotes. For some reason the XE YANG model developers starting from release 17.2.x dropped the double quotes in enum statements, which basically violates the RFC.

As a workaround I suggest edit file ~/.ydk//Cisco-IOS-XE-native.yang by taking enum values "+" and "-" into double quotes, then rerun your app. Once the YANG module is in the temp repo, it will not be overwritten. Please save the file(s) from temp repo into some safe place for future reuse as YANG model repo.