Undefined members of structures lead to huge export file sizes (e.g. xml)

[gynt]

Is your feature request related to a problem? Please describe. Large XML files, e.g. 3GB while the program is only 8MB, are caused in my case by the XML exporter writing many members of structures that are of datatype undefined. In my case, I know many of the sizes of structures, but I haven't documented all members yet. Likely, many of the undefined will turn about to be long arrays. Exporting "Data" is not really feasible this way.

Describe the solution you'd like I would like the option in the XML exporter to ignore undefined members of structures. Since every member also has an offset, I think this is no problem.

Describe alternatives you've considered Post-processing of the xml file. But then that means I would need to read in the file. The quick and dirty solution right now is to remove all lines in the xml file containing DATATYPE="undefined".

Additional context NA

[ghidra1]

What is your intended import side for the XML?

We may be able to omit undefined components. We certainly do not need them when importing back to Ghidra. The biggest issue is the IDA Pro import plugin - I am uncertain of its needs or sensitivity to omitting undefined filler components. It would be great if someone could experiment with this IDA Pro import when undefined components are dropped from the XML file. If there is no impact this should always be done.

The XML format is also quite out-of-date with Ghidra's structure attributes such as packing, alignment, etc.. Adding proper support for this may also impact compatibility with IDA Pro importer. It is primarily because of the IDA Pro side (Ghidra XML import/export) that our XML format has stagnated.

We also have a SARIF import/export which is more complete, but it too appears to be outputting undefined component details at this time. It may be more timely to correct this if you are simply round-tripping from/to Ghidra.

[ghidra1]

I have updated SARIF to only export define structure components. Change is in review and targets the master branch (11.3 release). I will explore XML change if I can find someone to test IDA Pro import with defined structure components only.

[gynt]

Thank you for looking into a solution

@ghidra1 I don't own IDA Pro so I am afraid I can't test the IDA Pro import functionality.

Currently I am using XML as a backup and also as a way of tracking changes and progress via a git repo which produces diffs. I will look into SARIF to see how to use it. I read so far that the output is json files so that sounds promising for my use case.

[ghidra1]

SARIF supports much more of Ghidra's markup capabilities while the XML is very dated and considered lossy in some cases.