CodecService fails to decode XML for Cisco XR Yang models when BGP and SNMP-IFMIB are present in the payload

[ldacol]

Issue tracker is ONLY used for reporting bugs. Please use the YDK Community for any support issues.

Expected Behavior

Current Behavior

CodecService fails to decode XML for Cisco XR Yang models when BGP and SNMP-IFMIB are present in the payload at the same time. Decode is successful when using either payload

Steps to Reproduce

Run the script below with the following payload referenced. The model is based on 7.6.2 but I believe it can be reproduced with any version.

Your Script

#!/Users/ldacol/ydk_vne/bin/ -i

import argparse
import json
import yaml
import xmltodict
import xml.etree.ElementTree
from lxml import etree
import re
import logging

from ydk.providers import CodecServiceProvider
from ydk.services import CodecService
from ydk.entity_utils import XmlSubtreeCodec
from ydk.path import NetconfSession
from ydk.path import Repository

CODEC = CodecService()

JSON_PROVIDER_CISCO_IOSXR_762 = CodecServiceProvider(type='json')

XML_PROVIDER_CISCO_IOSXR_762 = CodecServiceProvider(type='xml')

CODEC_XML = XmlSubtreeCodec()

JSON_PROVIDER_CISCO_IOSXR_762.initialize('iosxr-762',
    '/Users/ldacol/ydk_vne/lib/python3.8/site-packages/ydk/models/cisco_ios_xr_762/_yang')

XML_PROVIDER_CISCO_IOSXR_762.initialize('iosxr-762',
    '/Users/ldacol/ydk_vne/lib/python3.8/site-packages/ydk/models/cisco_ios_xr_762/_yang')

JSON_PROVIDER = JSON_PROVIDER_CISCO_IOSXR_762

XML_PROVIDER = XML_PROVIDER_CISCO_IOSXR_762

def enable_logging(level):
    log = logging.getLogger('ydk')
    log.setLevel(level)
    handler = logging.StreamHandler()
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    handler.setFormatter(formatter)
    log.addHandler(handler)
    return log

xml_payload_list = [
"""
  <bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XR-ipv4-bgp-cfg">
   <instance>
    <instance-name>default</instance-name>
   </instance>
  </bgp>
""",
"""
  <mib xmlns="http://cisco.com/ns/yang/Cisco-IOS-XR-snmp-agent-cfg">
   <interface-mib xmlns="http://cisco.com/ns/yang/Cisco-IOS-XR-snmp-ifmib-cfg">
    <interface-alias-long/>
    <interface-index-persistence/>
   </interface-mib>
  </mib>
""",
]

def xml_to_json(xml: str) -> str:
    try:
        model = CODEC.decode(XML_PROVIDER, xml)
        json = CODEC.encode(JSON_PROVIDER, model)
        return json
    except Exception as err:
        print(f"Exception: {err}")
        return None

if __name__ == "__main__":
    enable_logging(logging.INFO)
    json_payload = xml_to_json(xml_payload_list)
    if json_payload:
        if isinstance(json_payload, list):
            for j in json_payload:
                print(j)
        else:
            print(json_payload)
    else:
        print("Failed to transform XML payload to JSON")

Logs

Enable logging and post the logs below

(ydk_vne) ➜  GNSA python3 ydk_test.py
2022-11-15 10:00:15,650 - ydk - ERROR - Could not fetch schema node 'Cisco-IOS-XR-snmp-ifmib-cfg:interface-mib'
Exception:  Could not fetch schema node: Cisco-IOS-XR-snmp-ifmib-cfg:interface-mib
Failed to transform XML payload to JSON
(ydk_vne) ➜  GNSA

System Information

MacOs Ventura

(ydk_vne) ➜ GNSA python --version Python 3.8.15

(ydk_vne) ➜ GNSA pip freeze | grep ydk ydk @ file:///Users/ldacol/Code/GNSA/ydk-gen/gen-api/python/ydk/dist/ydk-0.8.6.3.tar.gz ydk-models-cisco-ios-xr-762 @ file:///Users/ldacol/Code/GNSA/ydk-gen/gen-api/python/cisco_ios_xr_762-bundle/dist/ydk-models-cisco-ios-xr-762-7.6.2.tar.gz ydk-models-ietf==0.1.6 ydk-models-juniper-junos-18-4R3 @ file:///Users/ldacol/Code/GNSA/ydk-gen/gen-api/python/juniper_junos_18_4R3-bundle/dist/ydk-models-juniper-junos-18-4R3-18.4.post3.tar.gz ydk-models-juniper-qfx-20-2R1 @ file:///Users/ldacol/Code/GNSA/ydk-gen/gen-api/python/juniper_qfx_20_2R1-bundle/dist/ydk-models-juniper-qfx-20-2R1-20.2.post1.tar.gz

[ygorelik]

I observed that the issue appears due to present of two codec providers in the same function. The workaround would be separate decode and encode in two different functions or change the encoding type on-the-fly. Example:

def xml_to_json(xml: str) -> str:
    codec = CodecService()
    provider = CodecServiceProvider()
    try:
        provider.encoding = EncodingFormat.XML
        model = codec.decode(provider, xml)

        provider.encoding = EncodingFormat.JSON
        json = codec.encode(provider, model)
        return json
    except Exception as err:
        print(f"Exception: {err}")
        return None

The result with this function implementation:

{
  "Cisco-IOS-XR-ipv4-bgp-cfg:bgp": {
    "instance": [
      {
        "instance-name": "default"
      }
    ]
  }
}

{
  "Cisco-IOS-XR-snmp-agent-cfg:mib": {
    "Cisco-IOS-XR-snmp-ifmib-cfg:interface-mib": {
      "interface-alias-long": [null],
      "interface-index-persistence": [null]
    }
  }
}