Possibility to save scene as "slim TXML"

[Stinkfist0]

Writing down an idea we discussed with @jonnenauha some weeks ago. Adding a possibility to save the scene in a slimmer format could had its advantages from time to time.

Opportunities for saving space when writing TXML:

1. omit default values of sync property (true) both for entities and components (for entities alternatively the sync value could be deducted from the entity ID),
2. omit component type names,
3. omit attributes with default values,
4. omit types of static attributes,
5. omit attribute names,
6. no indentation, and
7. write booleans as "1" and "0" instead of true and false. EDIT: 8. option to omit line endings also as suggested by @jonnenauha

A single entity example scene

<!DOCTYPE Scene>
<scene>
 <entity id="1" sync="true">
  <component typeId="20" type="Placeable" sync="true">
   <attribute value="-66.6853561,43.672966,-41.3164825,-12.0000153,-126.599899,0,1,1,1" type="Transform" id="transform" name="Transform"/>
   <attribute value="false" type="bool" id="drawDebug" name="Show bounding box"/>
   <attribute value="true" type="bool" id="visible" name="Visible"/>
   <attribute value="1" type="int" id="selectionLayer" name="Selection layer"/>
   <attribute value="" type="EntityReference" id="parentRef" name="Parent entity ref"/>
   <attribute value="" type="string" id="parentBone" name="Parent bone name"/>
  </component>
  <component typeId="26" type="Name" sync="true">
   <attribute value="FreeLookCameraSpawnPos" type="string" id="name" name="Name"/>
   <attribute value="" type="string" id="description" name="Description"/>
   <attribute value="" type="string" id="group" name="Group"/>
  </component>
 </entity>
</scene>

consisting of 985 characters (CRLF line endings) would become

<!DOCTYPE Scene>
<scene>
<entity id="1">
<component typeId="20">
<attribute value="-66.6853561,43.672966,-41.3164825,-12.0000153,-126.599899,0,1,1,1" id="transform"/>
</component>
<component typeId="26">
<attribute value="FreeLookCameraSpawnPos" id="name"/>
</component>
</entity>
</scene>

which is 301 characters, 30.56 % of the original TXML.

[jonnenauha]

Good work on describing the situation clearly. This would indeed be a nice win. If you want newlines and indentation you can just run a XML indentation tool of your choise on the file (eg. there is a sublime package for this).

I think omitting attributes that have default values should be made a new default. Yes I said it :) I see no point in writing the default values, no matter what the situation is. Here is my thinking, please leave a counter argument if you can think of one.

1. This would make .txml files more resilient to changes in component implementations. If Placeables default selection layer changes from 0 to 1 (will probably never happen, but as an example), all old txml would now load zero and eg. raycasting suddenly breaks. The user had no changed the selection layer, and expects the default to work!
2. The produced .txml is a lot easier to read, you can just see clearly what values deviate from the defaults.
3. This would absolutely be the biggest win in size, even if no other "slim" features would be implemented. From the usual Name, Placeable, Mesh, Light components the user usually changes values to 10-20% of the attributes.

For the rest like omitting comp/attribute names we could have Scene::SaveTxmlCustomized() taking the same bools as the current one and adding a bunch more. I think removing whitespace and newlines could be so that its not done in these function but a separate post processing func provided by CoreStringUtils or similar as QByteArray MinimizeTxml(const QByteArray &data, bool removeWhitespace, bool removeNewlines).

Edit: Now thinking of it the whitespace might be tricky to remove after the fact without knowing stuff. It could be made a int indentation = 2 in the custom save func.

[antont]

For completeness / just for info, here's the same as 'normal xml' / xml3d style xml

<!DOCTYPE Scene>
<scene>
  <entity id="1">
    <placeable>
      <transform value="-66.6853561,43.672966,-41.3164825,-12.0000153,-126.599899,0,1,1,1" />
    </placeable>
    <name>
       <name value="FreeLookCameraSpawnPos"/>
    </name>
  </entity>
</scene>

260 chars so not an awful lot of less than Stinkfist's example but somewhat less anyhow. And more human readable.

Obviously not as explicit machine readable as there are no component, attribute etc tags. A even more kind of normal xml would be:

<!DOCTYPE Scene>
<scene>
  <entity id="1">
    <placeable transform="-66.6853561,43.672966,-41.3164825,-12.0000153,-126.599899,0,1,1,1" />
    <name name="FreeLookCameraSpawnPos"/>
  </entity>
</scene>

That's down to 213 chars and very readable. Actually xml3d is more like this than the first example.

XML already itself has the <component attribute=value> structure so when there's no attribute metadata there's no need for an <attribute> tag .. but with this style of data docs the metadata for the type info etc. has to be gotten from outside (the component defs i figure).

[cadaver]

There are different tradeoffs involved, the question is mainly what do you want to give up:

• Possibility to store "unknown" components and load them later when the correct module is present (not possible if attribute types are not shown)
• Compatibility with existing TXML
• Ability to store per-attribute extra metadata (the extreme normalized form loses this)
• Sane for human to look at

@Stinkfist0's suggestion is fine for something that will only be processed by machines, and it retains compatibility. Saving default-valued attributes could be pretty much removed from everywhere without loss.

[jonnenauha]

If we do something to this, i would also suggest that we set a version from now on, eg. <scene version=1.0> or something, if we one day do very big changes we can route old versions to the correct reading code.