Add a way to automatically parse XML into a Dictionary

[tavurth]

Describe the project you are working on

A project sometimes has to fetch from legacy APIs, i.e. the U.S. Securities and Exchange Commission.

Describe the problem or limitation you are having in your project

Wow I'd forgotten how awful XML is to parse, and there's currently not an easy way to do that in Godot. The XMLParser is a little barebones and leaves it up to the user to parse the code into something usable (i.e Dictionary)

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I'd love to be able to run a single command to convert XML to Dictionary and vice-versa.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

XMLParser.parse({xml}) -> gives Dictionary

or perhaps:

XMLParser.to_dict({xml})

If this enhancement will not be used often, can it be worked around with a few lines of script?

It would take a whole addon to do this properly, although a GDNative script would be better as XML files can be very large.

Is there a reason why this should be core and not an add-on in the asset library?

Since we've got the XMLParser node already, it seems like this would be a good improvement.

[YuriSizov]

JSON is not a native Godot data format. Did you mean Dictionary?

[tavurth]

Yep, thank you corrected above.

[Calinou]

As far as I know, XML doesn't have a 1:1 mapping to JSON (and therefore Dictionary). For instance, a tag can contain text, have child tags and have attributes at the same time. The equivalent of this is not possible in a format like JSON (or not without an uncanny structure at least).

Also, XMLParser is considered to be on its way out and might no longer be exposed in 4.0. The editor will keep using it internally (it's needed for --doctool), but a full-fledged XML parser would be better implemented as an add-on.

[tavurth]

@Calinou after playing with the XML parser a bit it does seem less polished than the rest of the Godot ecosystem. However it's still invaluable for such projects, even if it takes a bit of time to learn.

If we could replace it with a full-fledged plugin it would be ideal but I think the use-cases of XML are fairly far-between in game development. Perhaps just shifting the existing code to a GDNative plugin would be enough to keep the Godot core slim.

[Calinou]

Perhaps just shifting the existing code to a GDNative plugin would be enough to keep the Godot core slim.

The editor can't rely on GDNative plugins for tools that will be used constantly by engine developers, especially since some form of global plugins aren't implemented yet.

However, unexposing the XMLParser class will allow us to not compile XMLParser in non-editor builds, making export template binaries a few kilobytes smaller.

[tavurth]

I see thank you for the insights! Hope we don't unexpose the XMLParser before there's a replacement available!

[Xrayez]

Hope we don't unexpose the XMLParser before there's a replacement available!

Note that unexposed classes can be further moved to modules such as Goost without much hassle (literally it would be a matter of copy-pasting the code, possibly further improved).

But for things like this, yeah perhaps GDScript or a GDNative plugin would be a better alternative, yet it would take time for XMLParser to be ported to those languages/stack.

[Gnumaru]

I've created an addon for parsing some data formats like xlsx, yaml and xml as raw data (dictionaries, arrays etc).

https://godotengine.org/asset-library/asset/3048

It is an import plugin intended to be used in the editor only because it relies on python for data extraction. But if your application is meant for desktop, the python script that converts xml into json can be used independently of the addon. It just walks the node tree and creates a nested dictionary structure similar to this.

{"tag": "my-xml-tag",
 "attributes": {"a": "1", "b": "2", "c": "3"},
 "children": [
    "raw text here",
    {"tag": "an-empty-tag", "attributes": {}, "children": []},
    "more raw text here"]}

It is a very clunky structure but it is the only way of keeping the node children ordered. But if child order doesnt matter the structure could be much better by grouping together into arrays children nodes of the same tag and putting the xml node properties as properties of the dictionary.

{"tag": "my-xml-tag",
 "a": "1",
 "b": "2",
 "c": "3",
 "text": ["raw text here", "more raw text here"]
 "an-empty-tag": [
    {"tag": "an-empty-tag"}]}