Figure out deserialization mapping of non-public fields

[ztzg]

Some YAML files refer to internal properties which are not directly exposed in the .NET API, but that data is most probably necessary to properly implement some of the public method or global behaviours.

As an example, ParticleSystem only exposes a small set of properties, but its serialization includes a large list of "modules" which determine its exact behaviour at runtime:

InitialModule:
  enabled: 1
  startLifetime:
    scalar: 3.5
    maxCurve:
      serializedVersion: 2
      m_Curve:
      - time: 0
        value: 1
        inSlope: 0
        outSlope: 0
        tangentMode: 0
      - time: 1
[…]
SubModule:
  enabled: 0
  subEmitterBirth: {fileID: 0}
  subEmitterBirth1: {fileID: 0}
  subEmitterCollision: {fileID: 0}
  subEmitterCollision1: {fileID: 0}
  subEmitterDeath: {fileID: 0}
  subEmitterDeath1: {fileID: 0}

Note the elision marker. The *Module subtree actually spans 1013 lines!

[bobsummerwill]

Yuck. Got some more deserialization working to get the fileIDs at least read in, but it's an abomination!

https://github.com/kitsilanosoftware/Bosphorus/commit/a369f61f457c7dc7506c545a4913ec13c1181d4c

https://github.com/kitsilanosoftware/BosphorusEngine/commit/1d8ef29bb2d2bb03c1489e1f2f8db6bf74330d6f

I think that the file-format that Unity are using here is really non-idiomatic YAML. In particular, this in GameObject:

m_Component:

• 4: {fileID: 1099934815}
• 20: {fileID: 1099934814}
• 92: {fileID: 1099934813}
• 124: {fileID: 1099934812}
• 81: {fileID: 1099934811}

Actually appears to be a sequence of mappings of mappings, and the outer mapping is NOT using a consistent key name. The key value (the object type) is actually being used as a key. Maybe this is some manifestation of their underlying C++ code? Whatever the case, it is just horrible.

I did something horrific to get the data read in, and will have to try to simplify it now I now what the events are being mapped as.

Or maybe I will just shelve that horror for now and move onto getting the runtime fixing up the references and actually running a bit.

I see that the YAML also has fileIDs which aren't listed in the YAML itself, which are presumably for external assets, like the script code, meshes, etc.

I found something online describing the hashing used for fileID, which is some hash of the object type name and object name concatenated.

[ztzg]

Hi Bob,

I'll do something quick-and-dirty for now, and then clean it up later.

I want to get a running tiny Unity scene with a MVVM abstraction soon, based on deserializing EmptyUnity2DProject, which just has a camera and a cube and those various "settings" objects.

Sounds good!

Yuck. Got some more deserialization working to get the fileIDs at least read in, but it's an abomination!

https://github.com/kitsilanosoftware/Bosphorus/commit/a369f61f457c7dc7506c545a4913ec13c1181d4c

https://github.com/kitsilanosoftware/BosphorusEngine/commit/1d8ef29bb2d2bb03c1489e1f2f8db6bf74330d6f

Well, one step at a time :) This doesn't look bad at a glance; the only mistake would be to not schedule a clean up once the required mechanics are better understood.

I think that the file-format that Unity are using here is really non-idiomatic YAML. In particular, this in GameObject:

m_Component:

• 4: {fileID: 1099934815}
• 20: {fileID: 1099934814}
• 92: {fileID: 1099934813}
• 124: {fileID: 1099934812}
• 81: {fileID: 1099934811}

Actually appears to be a sequence of mappings of mappings, and the outer mapping is NOT using a consistent key name. The key value (the object type) is actually being used as a key. Maybe this is some manifestation of their underlying C++ code? Whatever the case, it is just horrible.

C++: most probably. The serialization format doesn't seem to have been "designed," but rather to stem from their internal object model.

I did something horrific to get the data read in, and will have to try to simplify it now I now what the events are being mapped as.

Or maybe I will just shelve that horror for now and move onto getting the runtime fixing up the references and actually running a bit.

I, for one, would try to get something useful running ASAP, because I'm sure you will discover some other warts while doing so. Refactoring is likely to be more successful with a better knowledge of the graph :)

I see that the YAML also has fileIDs which aren't listed in the YAML itself, which are presumably for external assets, like the script code, meshes, etc.

I found something online describing the hashing used for fileID, which is some hash of the object type name and object name concatenated.

Really? "fileID"s seem to be 32-bit integers; they would need to somehow cater for collisions. Or are you talking about the other, longer hashes?

Cheers, -D

[bobsummerwill]

Ah, yes, I misspoke slightly. The reference I saw was for this "fileID" only ...

m_Script: {fileID: -1167294237, guid: 2b16a1acf52a2a64e916f8a9e6d5df31, type: 3}

http://forum.unity3d.com/threads/yaml-fileid-hash-function-for-dll-scripts.252075/