search

ssato / python-anyconfig

Python library provides common APIs to load and dump configuration files in various formats
MIT License
278 stars 31 forks source link

xml parsing fails when root key has multiple attributes #63

Closed ajays20078 closed 7 years ago

ajays20078 commented 7 years ago

Example of xml:

<persistence version="2.0"
    xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">

    <persistence-unit name="separator" transaction-type="RESOURCE_LOCAL">
        <provider>org.hibernate.ejb.HibernatePersistence</provider>
        <class>com.example.separator.domain.Job</class>
        <class>com.example.separator.domain.Module</class>
        <class>com.example.separator.domain.Condition</class>
        <class>com.example.separator.domain.ConditionValue</class>
        <class>com.example.separator.domain.Coupon</class>
        <class>com.example.separator.domain.CouponCondition</class>
        <class>com.example.separator.domain.CouponInstance</class>
        <class>com.example.separator.domain.CouponInstanceUsage</class>
        <class>com.example.separator.domain.CouponSchedule</class>
        <class>com.example.separator.domain.CustomCouponAttribute</class>
                <class>com.example.separator.domain.CouponInstanceUser</class>
        <class>com.example.separator.domain.CouponRule</class>
        <class>com.example.separator.domain.CouponRuleLeftOperand</class>
        <class>com.example.separator.domain.CouponRuleItem</class>
        <exclude-unlisted-classes>true</exclude-unlisted-classes>
    </persistence-unit>

</persistence>

anyconfig.load is resulting in a dictionary with weird keys.

{'{http://java.sun.com/xml/ns/persistence}persistence': {'@attrs': {'version': '2.0', '{http://www.w3.org/2001/XMLSchema-instance}schemaLocation': 'http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd'}, '@children': [{'{http://java.sun.com/xml/ns/persistence}persistence-unit': {'@attrs': {'name': 'separator', 'transaction-type': 'RESOURCE_LOCAL'}, '@children': [{'{http://java.sun.com/xml/ns/persistence}provider': {'@text': 'org.hibernate.ejb.HibernatePersistence'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.Job'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.Module'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.Condition'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.ConditionValue'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.Coupon'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponCondition'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponInstance'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponInstanceUsage'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponSchedule'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CustomCouponAttribute'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponInstanceUser'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponRule'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponRuleLeftOperand'}}, {'{http://java.sun.com/xml/ns/persistence}class': {'@text': 'com.example.separator.domain.CouponRuleItem'}}, {'{http://java.sun.com/xml/ns/persistence}exclude-unlisted-classes': {'@text': 'true'}}]}}]}}

and when you check the keys of the dict, it shows them as

dict_keys(['persistence', '{http://java.sun.com/xml/ns/persistence}persistence'])

Whereas when you load the same using xmltodict, it handles it properly.

>>> with open('1.xml') as fh:
...   content = xmltodict.parse(fh.read())
... 
>>> content
OrderedDict([('persistence', OrderedDict([('@version', '2.0'), ('@xmlns', 'http://java.sun.com/xml/ns/persistence'), ('@xmlns:xsi', 'http://www.w3.org/2001/XMLSchema-instance'), ('@xsi:schemaLocation', 'http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd'), ('persistence-unit', OrderedDict([('@name', 'separator'), ('@transaction-type', 'RESOURCE_LOCAL'), ('provider', 'org.hibernate.ejb.HibernatePersistence'), ('class', ['com.example.separator.domain.Job', 'com.example.separator.domain.Module', 'com.example.separator.domain.Condition', 'com.example.separator.domain.ConditionValue', 'com.example.separator.domain.Coupon', 'com.example.separator.domain.CouponCondition', 'com.example.separator.domain.CouponInstance', 'com.example.separator.domain.CouponInstanceUsage', 'com.example.separator.domain.CouponSchedule', 'com.example.separator.domain.CustomCouponAttribute', 'com.example.separator.domain.CouponInstanceUser', 'com.example.separator.domain.CouponRule', 'com.example.separator.domain.CouponRuleLeftOperand', 'com.example.separator.domain.CouponRuleItem']), ('exclude-unlisted-classes', 'true')]))]))])
>>> dict(content).keys()
dict_keys(['persistence'])
ssato commented 7 years ago

Thanks a lot for your report! I'll look into this.

ssato commented 7 years ago

How about the master head after the commit de1267b ? It looks fine as far as I tested.

In [1]: import os,sys; sys.path.insert(0, os.curdir); import anyconfig

In [2]: xml_s = open("/tmp/0.xml").read()

In [3]: from __future__ import print_function

In [4]: print(xml_s)
<persistence version="2.0"
        xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">

        <persistence-unit name="separator" transaction-type="RESOURCE_LOCAL">
                <provider>org.hibernate.ejb.HibernatePersistence</provider>
                <class>com.example.separator.domain.Job</class>
                <class>com.example.separator.domain.Module</class>
                <class>com.example.separator.domain.Condition</class>
                <class>com.example.separator.domain.ConditionValue</class>
                <class>com.example.separator.domain.Coupon</class>
                <class>com.example.separator.domain.CouponCondition</class>
                <class>com.example.separator.domain.CouponInstance</class>
                <class>com.example.separator.domain.CouponInstanceUsage</class>
                <class>com.example.separator.domain.CouponSchedule</class>
                <class>com.example.separator.domain.CustomCouponAttribute</class>
                <class>com.example.separator.domain.CouponInstanceUser</class>
                <class>com.example.separator.domain.CouponRule</class>
                <class>com.example.separator.domain.CouponRuleLeftOperand</class>
                <class>com.example.separator.domain.CouponRuleItem</class>
                <exclude-unlisted-classes>true</exclude-unlisted-classes>
        </persistence-unit>

</persistence>

In [5]: xml_d = anyconfig.loads(xml_s, ac_parser="xml")

In [6]: xml_d
Out[6]:
{'persistence': {'@attrs': {'version': '2.0',
   'xmlns': 'http://java.sun.com/xml/ns/persistence',
   'xmlns:xsi': 'http://www.w3.org/2001/XMLSchema-instance',
   '{http://www.w3.org/2001/XMLSchema-instance}schemaLocation': 'http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd'},
  'persistence-unit': {'@attrs': {'name': 'separator',
    'transaction-type': 'RESOURCE_LOCAL'},
   '@children': [{'provider': 'org.hibernate.ejb.HibernatePersistence'},
    {'class': 'com.example.separator.domain.Job'},
    {'class': 'com.example.separator.domain.Module'},
    {'class': 'com.example.separator.domain.Condition'},
    {'class': 'com.example.separator.domain.ConditionValue'},
    {'class': 'com.example.separator.domain.Coupon'},
    {'class': 'com.example.separator.domain.CouponCondition'},
    {'class': 'com.example.separator.domain.CouponInstance'},
    {'class': 'com.example.separator.domain.CouponInstanceUsage'},
    {'class': 'com.example.separator.domain.CouponSchedule'},
    {'class': 'com.example.separator.domain.CustomCouponAttribute'},
    {'class': 'com.example.separator.domain.CouponInstanceUser'},
    {'class': 'com.example.separator.domain.CouponRule'},
    {'class': 'com.example.separator.domain.CouponRuleLeftOperand'},
    {'class': 'com.example.separator.domain.CouponRuleItem'},
    {'exclude-unlisted-classes': 'true'}]}}}

In [7]:
ssato commented 7 years ago

I forgot about the attributes, I'll fix that also.

'{http://www.w3.org/2001/XMLSchema-instance}schemaLocation': 'http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd'},
ssato commented 7 years ago

Fixed attribute cases in the commit 9e45085.

        ...
In [5]: xml_d = anyconfig.loads(xml_s, ac_parser="xml")

In [6]: xml_d
Out[6]:
{'persistence': {'@attrs': {'version': '2.0',
   'xmlns': 'http://java.sun.com/xml/ns/persistence',
   'xmlns:xsi': 'http://www.w3.org/2001/XMLSchema-instance',
   'xsi:schemaLocation': 'http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd'},
  'persistence-unit': {'@attrs': {'name': 'separator',
    'transaction-type': 'RESOURCE_LOCAL'},
   '@children': [{'provider': 'org.hibernate.ejb.HibernatePersistence'},
    {'class': 'com.example.separator.domain.Job'},
    {'class': 'com.example.separator.domain.Module'},
    {'class': 'com.example.separator.domain.Condition'},
    {'class': 'com.example.separator.domain.ConditionValue'},
    {'class': 'com.example.separator.domain.Coupon'},
    {'class': 'com.example.separator.domain.CouponCondition'},
    {'class': 'com.example.separator.domain.CouponInstance'},
    {'class': 'com.example.separator.domain.CouponInstanceUsage'},
    {'class': 'com.example.separator.domain.CouponSchedule'},
    {'class': 'com.example.separator.domain.CustomCouponAttribute'},
    {'class': 'com.example.separator.domain.CouponInstanceUser'},
    {'class': 'com.example.separator.domain.CouponRule'},
    {'class': 'com.example.separator.domain.CouponRuleLeftOperand'},
    {'class': 'com.example.separator.domain.CouponRuleItem'},
    {'exclude-unlisted-classes': 'true'}]}}}

In [7]:
ajays20078 commented 7 years ago

Will check and revert.

ajays20078 commented 7 years ago

Yeah, this works. Thanks for the quick fix.