CZ-NIC / yangson

GNU Lesser General Public License v3.0
53 stars 20 forks source link

support XML encoding #20

Closed kwatsen closed 3 years ago

kwatsen commented 5 years ago

As discussed at the 103 social, it would would be excellent to also support XML-encoded instance data.

My project uses supports both XML and JSON. I imagine many projects are this way...

kwatsen commented 5 years ago

One issue that may be encountered while trying to address this issue is which of the several available Python modules from XML to use.

Of course, when the Yangson API is importing/exporting a string containing YANG-compliant XML, this isn't an issue, as Yangson can do internally whatever it needs to do.

The issue is more for when the Yangson API is importing/exporting raw values. This is an issue only because no Python module for XML seems to work seems to give the expected result. Some modules looked at include:

For instance, the JSON:

{
  "my-module:input" : {
    "foobar" : [null]
  }
}

becomes (with the standard json module):

{
  'my-module:input': {
    'foobar': [None]
  }
}

Whereas the XML:

<input xmlns="https://example.com/my-module">
  <foobar/>
</input>

becomes (with xmltodict and some parameters to map namespaces into prefixes):

OrderedDict([
  (
    'my-module:input', 
    OrderedDict([
      (
        'my-module:foobar', None
      )
    ])
  )
])

which, when loaded into Yangson, using SchemeNode.from_raw(), produces the following error:

    sn.from_raw(req_input)
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemanode.py", line 446, in from_raw
res[ch.iname()] = ch.from_raw(rval[qn], npath)
  File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemanode.py", line 834, in from_raw
    raise RawTypeError(jptr, self.type.yang_type() + " value")
yangson.exceptions.RawTypeError: [/my-module:foobar] expected empty value

Results from lxml.objectify and untangle produce results that are even further from pure Python structures (i.e., without use of special module-specific classes).

This makes me wonder if if might not be possible to "simple" YANG-based XML to/from pure Python structure that, by nature of being solely for YANG-modeled objects, can do the right thing...

That said, the easiest/best approach should come from a convertor that is schema-driven, but then this is exactly where Yangson can fit it. For instance, SchemaNode.obj_to_xml(python_object) -> xml_string and `SchemaNode.xml_to_obj(xml_string) -> python_object.

Actually, it seems that, if these two methods were provided, then this issue could be closed, as these methods could be use in conjunction with the existing to_raw and from_raw methods for functional albeit less efficient XML support, which might be a sweet spot.

llhotka commented 5 years ago

JSON encoding according to RFC 7951 requires schema-driven parsing, so it is not surprising that generic XML-JSON convertors fall short.

The best approach would indeed be to implement methods like SchemaNode.from_xml and InstanceNode.to_xml.

HRogge commented 4 years ago

I am looking into this issue at the moment... the main trouble I see is that XML has no "array" element... which means you cannot detect from the XML data alone if something is an "object" or a "list". This pretty much prevents an easy XML to JSON/python-raw-object conversion.

llhotka commented 4 years ago

Right, but you can get the info from the data model. I don't think adding support for XML representation would be too difficult.

HRogge commented 4 years ago

I have the "to_xml()" part done based on the python xml.etree package... from_xml() is nearly done too. There are three remaining issues...

HRogge commented 4 years ago

I got feedback from the IETF netconf group about the anyxml/anydata problem...

they suggest dropping anyxml completely (or restricting it to one encoding) and allowing anydata only when a corresponding YANG schema is known.

HRogge commented 4 years ago

I have send a pull request for basic XML support: #51 I think it should work (except for the problems discussed above).

kwatsen commented 3 years ago

Closing this issue as general XML support is now being added (e.g., Pull #81) and we've now moved to filing issues for specific XML-related issues.