Mesh Assets

[ambethia]

How difficult do you think it would be to also extract mesh assets? I'm going to have a stab at it myself, but I'm not all that familiar with how Unity handles these internally. I'm thinking if I can save out the mesh data in Unity's native format, I should be able to add them to a scene and use an editor script to export an Obj or similar?

[ambethia]

I created a MeshHandler class, but in trying to get the m_Name property, I'm bumping up against:

[warning] AssetExtractor: Can't deserialize Object #2144176810 (ClassID: 43, Class: Mesh), caused by info.ata4.unity.serdes.DeserializerException: Can't read value of field m_DataSize
at info.ata4.unity.serdes.Deserializer.readField(Deserializer.java:98)
at info.ata4.unity.serdes.Deserializer.readObject(Deserializer.java:84)
at info.ata4.unity.serdes.Deserializer.readComplex(Deserializer.java:183)
at info.ata4.unity.serdes.Deserializer.readFieldValue(Deserializer.java:112)
at info.ata4.unity.serdes.Deserializer.readField(Deserializer.java:96)
at info.ata4.unity.serdes.Deserializer.deserialize(Deserializer.java:60)
at info.ata4.unity.extract.AssetExtractor.extract(AssetExtractor.java:174)
at info.ata4.unity.DisUnity.processAsset(DisUnity.java:142)
at info.ata4.unity.DisUnity.processAsset(DisUnity.java:85)
at info.ata4.unity.DisUnity.processAssetBundle(DisUnity.java:214)
at info.ata4.unity.DisUnity.processAssetBundle(DisUnity.java:226)
at info.ata4.unity.DisUnity.run(DisUnity.java:250)
at info.ata4.unity.cli.DisUnityCli.main(DisUnityCli.java:55)
Caused by: java.io.EOFException
at info.ata4.util.io.ByteBufferInput.readFully(ByteBufferInput.java:35)
at info.ata4.util.io.DataInputWrapper.readFully(DataInputWrapper.java:29)
at info.ata4.unity.serdes.SerializedInput.readByteArray(SerializedInput.java:89)
at info.ata4.unity.serdes.Deserializer.readArray(Deserializer.java:196)
at info.ata4.unity.serdes.Deserializer.readComplex(Deserializer.java:180)
at info.ata4.unity.serdes.Deserializer.readFieldValue(Deserializer.java:112)
at info.ata4.unity.serdes.Deserializer.readField(Deserializer.java:96)
... 12 more

At this point, I don't know enough about the internals to keep going. I'll keep poking at it.

[ata4]

The deserializer is still far from being perfect and may fail frequently, especially on standalone asset files. There are still some quirks to be reverse-engineered before it can work reliably on more complex structures, such as mesh objects. But yes, I'm planning to add support for static meshes sooner or later.

[ambethia]

Thanks!

[andrewpros]

Hi. I have the same problem, but manged to narrow it down.

Sample MeshHandler

public class MeshHandler extends ExtractHandler {

    @Override
    public String getClassName() {
        return "Mesh";
    }

    @Override
    public String getFileExtension() {
        return "mesh";
    }

    @Override
    public void extract(ObjectPath path, UnityObject obj) throws IOException {
        String name = obj.getValue("m_Name");
        String data = "";
        writeFile(data.getBytes(), path.pathID, name);
    }

}

And here is the Mesh structure dump

Mesh
  string m_Name
    Array Array
      SInt32 size
      char data
  vector m_SubMeshes
    Array Array
      SInt32 size
      SubMesh data
        UInt32 firstByte
        UInt32 indexCount
        SInt32 topology
        UInt32 firstVertex
        UInt32 vertexCount
        AABB localAABB
          Vector3f m_Center
            float x
            float y
            float z
          Vector3f m_Extent
            float x
            float y
            float z
  BlendShapeData m_Shapes
    vector vertices
      Array Array
        SInt32 size
        BlendShapeVertex data
          Vector3f vertex
            float x
            float y
            float z
          Vector3f normal
            float x
            float y
            float z
          Vector3f tangent
            float x
            float y
            float z
          UInt32 index
    vector shapes
      Array Array
        SInt32 size
        MeshBlendShape data
          UInt32 firstVertex
          UInt32 vertexCount
          bool hasNormals
          bool hasTangents
    vector channels
      Array Array
        SInt32 size
        MeshBlendShapeChannel data
          string name
            Array Array
              SInt32 size
              char data
          UInt32 nameHash
          SInt32 frameIndex
          SInt32 frameCount
    vector fullWeights
      Array Array
        SInt32 size
        float data
  vector m_BindPose
    Array Array
      SInt32 size
      Matrix4x4f data
        float e00
        float e01
        float e02
        float e03
        float e10
        float e11
        float e12
        float e13
        float e20
        float e21
        float e22
        float e23
        float e30
        float e31
        float e32
        float e33
  UInt8 m_MeshCompression
  UInt8 m_StreamCompression
  bool m_IsReadable
  bool m_KeepVertices
  bool m_KeepIndices
  vector m_IndexBuffer
    Array Array
      SInt32 size
      UInt8 data
  vector m_Skin
    Array Array
      SInt32 size
      BoneInfluence data
        float weight[0]
        float weight[1]
        float weight[2]
        float weight[3]
        SInt32 boneIndex[0]
        SInt32 boneIndex[1]
        SInt32 boneIndex[2]
        SInt32 boneIndex[3]
  VertexData m_VertexData
    UInt32 m_CurrentChannels
    UInt32 m_VertexCount
    vector m_Channels
      Array Array
        SInt32 size
        ChannelInfo data
          UInt8 stream
          UInt8 offset
          UInt8 format
          UInt8 dimension
    vector m_Streams
      Array Array
        SInt32 size
        StreamInfo data
          UInt32 channelMask
          UInt32 offset
          UInt8 stride
          UInt8 dividerOp
          UInt16 frequency
    TypelessData m_DataSize
      SInt32 size
      UInt8 data
  CompressedMesh m_CompressedMesh
    PackedBitVector m_Vertices
      UInt32 m_NumItems
      float m_Range
      float m_Start
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_UV
      UInt32 m_NumItems
      float m_Range
      float m_Start
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_BindPoses
      UInt32 m_NumItems
      float m_Range
      float m_Start
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_Normals
      UInt32 m_NumItems
      float m_Range
      float m_Start
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_Tangents
      UInt32 m_NumItems
      float m_Range
      float m_Start
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_Weights
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_NormalSigns
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_TangentSigns
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_BoneIndices
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_Triangles
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
    PackedBitVector m_Colors
      UInt32 m_NumItems
      vector m_Data
        Array Array
          SInt32 size
          UInt8 data
      UInt8 m_BitSize
  AABB m_LocalAABB
    Vector3f m_Center
      float x
      float y
      float z
    Vector3f m_Extent
      float x
      float y
      float z
  SInt32 m_MeshUsageFlags

As you can see the Mesh type has some fields that disunity have no support for.

The interesting function is readComplex in Deserializer.java

This function decides how to read some complex types, for Mesh the first problem is VertexData, you can add it to this method

case "VertexData":

but whatever you will try, it will fail, for return readArray(field); it will be

[warning] AssetExtractor: Can't extract Object #22 (ClassID: 43, Class: Mesh), caused by java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
    at info.ata4.unity.serdes.Deserializer.readArray(Deserializer.java:192)
    at info.ata4.unity.serdes.Deserializer.readComplex(Deserializer.java:182)
    at info.ata4.unity.serdes.Deserializer.readFieldValue(Deserializer.java:112)

So it is quite close to working, but my knowledge of java is too little to make it work, i just don't know how to read VertexData without error, it is quite different structure than others.

Maybe there will be also some other fields to fix like this for Mesh (for example PackedBitVector ) i don't know, can't get past VertexData error, but hey i think this will help to make it happen.

As a side note, it seems it would be impossible to get the original Mesh as an ready model for unity, so i think it only needs to extract vertex data, maybe also UV, then convert it to the simples 3d format like .x and then use an x to fbx converter.

Hope this helps, what do you think?

[ata4]

readComplex() handles a few special cases where the data is deserialized into more convenient/efficient structures. Which means that every case in readComplex() can also be deserialized with readObject(), including VertexData.

The problem with the deserialization process is that it's designed to be fast, not robust. Every byte of every object must be read exactly like Unity does or an exception most likely will be thrown. Or the objects will have lots of invalid values at best. And apparently, there are a few weird special cases where the exact reading method depends on flags that haven't been reverse-engineered yet, I figured out only one by myself so far. Mesh objects probably use some more of these, that's why DisUnity can't read them right now.

I'm also a bit worried about the code quality. The way I'm deserializing Unity objects in Java right now just doesn't feel "right". It probably will be refactored later.

[andrewpros]

Yeah i know every byte needs to be read, but we don't need all of them, can't use such data in unity anyway.

As for the code quality i would say, first let it work, then refactor.

Still, best software i have found to reverse unity data/obtain information that no one even mentions, thank you for that and keep up the good work.

[ata4]

I think I found the bug that prevented the deserialization of mesh objects. So writing a mesh handler should now be possible, at least in theory.

[andrewpros]

Yes, now it works, but i have no clue what data to extract and what format to save it the way it would be loadable by some 3d tools to preview it.

Maybe the vertex data would be enough.

It cannot be saved in unity format so it either need to be an 3d model file generator (like fbx, obj) or just very simple 3d format (.x ?) that can be converted using 3rd party tools to unity accepted one.

[ambethia]

I would be so happy for just obj.

http://en.wikipedia.org/wiki/Wavefront_.obj_file#File_format

I haven't had a chance yet to play with this since the serializer was patched, but I'm happy to have a stab at writing an OBJ exporter if we can iterate the vertex data now.

[ata4]

Actually, I just recently added a mesh handler that writes .obj files, it's available in version 0.1.3. It works with Unity 4.0+ only, but it's a start and can be extended to support older formats, too.

[andrewpros]

I can confirm that 0.1.3 and your MeshHandler is working flawlessly, unity opens the .obj files without a problem, you are the man, thanks!