strange tag while writing xml with nullValue

[groneveld]

while writing xml with:

df.repartition(1).write.format("xml")\
    .option("rootTag", "DataContainer")\
    .option("rowTag", "Свед")\
    .mode("overwrite")\
    .save(data_path))

get strange tag in resulting xml for elements, which were passed with Null values. for example:

<БазСвед>
    <Фамилия>some text</Фамилия>
     ....
</БазСвед>
<СведЕСИА/>

Check the last row. also, pay attention at slash and its position in tag.

Using: spark-xml_2.12-0.14.0.jar txw2-2.3.4.jar scala 2.12.15 spark 3.3.1

[srowen]

What is strange about this?

[groneveld]

@srowen this is neither open nor close tag. it's like alone. If we have some data inside, tags should look like: <БазСвед> # open .... </БазСвед> #close

and none of them if there is null.

[groneveld]

@srowen it happens each time we have nested stucture with all null nested elements. So, that's how the parent element is written. I guess, it should not be written at all.

[srowen]

All the tags you show are closed, I'm not sure what you're referring to?

[groneveld]

@srowen

<БазСвед>
    <Фамилия>some text</Фамилия>
     ....
</БазСвед>
<СведЕСИА/>

<СведЕСИА/> - this one from my example. it's alone

[srowen]

No, that is a closed tag. It's empty. <foo/> is the same as <foo></foo> in XML.

[groneveld]

@srowen ok, but is there any way it won't be written at all as it is empty? all nested tags are not written and that's what we expect. but why this parent tag is still written? even being empty

[srowen]

I don't think it's possible, no. You can post-process the XML how you like though.