Closed schneifejan closed 8 months ago
I think this only works if there are only attributes, no children, if you have attributePrefix == "". The schema becomes ambiguous when it goes back to read attributes vs children. I would just not set this option, and rename attribute fields as you see fit.
It's probably hard but possible to 'fix' the behavior in the code, but this library is not developed anymore now that it's in Spark. I'd look at a PR but otherwise not going to change the beahvior.
Thanks @srowen for explaining the behavior and the suggested workaround. As the library is not developed anymore I'd refrain from working on a PR. I'll close the issue then :)
Hi, I try to create a
DataFrame
without the defaultAttributePrefix
_
.Given the following XML sample data:
If I create a spark
DataFrame
without theAttributePrefix
option:the XML sample data is parsed as expected:
However, if I create a spark
DataFrame
with theAttributePrefix
option set to""
:the child-tags of
bar
are allNULL
.Does anybody have an idea what I'm doing wrong? Any help is highly appreciated 👍
Kind regards Felix
versions
Python 3.9 spark-xml: 0.17.0 pyspark 3.5.0