ILCPP support

[juliolitwin]

Yo,

There are no new commits for months, has development been stopped?

Cheers!

[jamescourtney]

Hi There --

The project is active-ish. It's something I'm interested in, but have relatively little time to devote to.

[juliolitwin]

Have you ever done any tests using unity il2cpp?

[jamescourtney]

I have not. What did you have in mind? I’ve never seen il2cpp before.

[juliolitwin]

Oh, I see...

You to can see some explanation here: https://docs.unity3d.com/Manual/IL2CPP.html

Usually to avoid problems with il2cpp, it is avoiding IL and reflections. ZeroFormatter supports il2cpp but unfortunately it is a library already "abandoned" by the author, it is only paying attention to the MessagePack.

[jamescourtney]

From my quick reading, using FlatSharp "out of the box" with ILcpp probably won't be a great experience, since Flatsharp generates C# code at runtime and then pushes that through Roslyn to generate IL, which would then need to be translated to CPP and recompiled to your target code.

However, it does expose some hooks to let you get at the generated assembly or generated C# code (which is not super pretty, but you should be able to make sense of it). Try something like this:

var serializer = FlatBufferSerailizer.Default.Compile<MyTable>();
string cSharp = serializer.CSharp;
ImmutableArray<byte> assembly = serializer.AssemblyBytes;

Both of those should give you something you can push through ILCPP and at least figure out if it's viable. One thing I can think of would be to extend the work I just did to support FBS files to also generate serializers as well as schema. This would require a bit of thought, but it's not a bad idea if there's a viable use case for it

[jamescourtney]

My thoughts on this have congealed a little bit, and I think there are some interesting features that I can build with codegen at buildtime instead of runtime. The large picture is that Flatsharp will support "easy mode" stuff at runtime (ie, your custom classes tagged with attributes), and advanced stuff at build time (pregenerated seriaizers, grpc, fbs schemas, etc).

Adding build-time serializers to support unity is a nice feature to add. I'm currently working on this, and should be done in a couple of days. I've never used Unity -- can I ask you to be a beta tester?

[juliolitwin]

Yo,

Now that I came to see your message ...

I'm still on vacation, but I can check it out later, it will be great to have support for il2cpp.

[jamescourtney]

Sounds good. Enjoy your vacation. Package version 1.2 is published with (what I hope is) support for this. You can see examples of using FBS schemas under the samples directory.

[juliolitwin]

Well, I want to test FlatSharp now. Just a question, in my current serializer, I use an interface like above:

public interface IPacket
{
      int OpCode {get; }
}

and my packets are:

public struct DummyPacket: IPacket
{
       public int OpCode => 1;
       public ItemData Item {get; set;}
}

And with serializer I can serialize and deserialize for example Lib.Serializer<IPacket>() and Lib.Deserializer<IPacket>(). Does FlatSharp support poliformism? I'm not have experience with FlatBuffers.

Cheers.

[jamescourtney]

Since you're using ILCPP, you'll want to generate all your code at build time rather than runtime. You'll need these two packages: FlatSharp.Compiler (Version = 2.0.2) FlatSharp.Runtime (Version = 2.0.0)

Then, you'll need to declare an FBS schema:

namespace SomeNamespace;

table DummyPacket (PrecompiledSerializer:greedymutable) {
   OpCode:int = 1;
   Item:ItemData;
}

table ItemData {
   // whatever this definition is. You'll need to define ItemData in a way that FlatSharp can understand.
   // FlatBuffers is not a serialization format that accepts arbitrary data and types.
   Data:string;
}

Next, add this to the csproj where you installed FlatSharp.compiler:

<ItemGroup>
    <FlatSharpSchema Include="MySchema.fbs" />
</ItemGroup>

This will produce a file with definitions for ItemData and DummyPacket, as well as a serializer for DummyPacket (available under DummyPacket.Serializer).

Now, to add your interfaces. FlatSharp's compiler generates partial classes, so you can simply add this to a file that you maintain:

namespace SomeNamespace
{
    // Add IPacket to the generated definition for DummyPacket.
    // If OpCode does not need to be serialized, then you can just add the read-only property here too.
   public partial class DummyPacket : IPacket { }
}

Take a few minutes and read the FlatSharp samples, as well as the basic documentation from Google's flatbuffers site (both linked on the index page of this repo). I think that will give you some more clarity about pros and cons of FlatBuffers as a choice for serialization format. There are some topics in FlatSharp we haven't touched on here either, such as deserialization mode (lazy, greedy, etc) that the samples will clarify.

The best sample for this is here

[jamescourtney]

Have you had any luck? Anything (not) working?

[jamescourtney]

Closing due to inactivity.

[shtse8]

Since you're using ILCPP, you'll want to generate all your code at build time rather than runtime. You'll need these two packages: FlatSharp.Compiler (Version = 2.0.2) FlatSharp.Runtime (Version = 2.0.0)

Then, you'll need to declare an FBS schema:

namespace SomeNamespace;

table DummyPacket (PrecompiledSerializer:greedymutable) {
   OpCode:int = 1;
   Item:ItemData;
}

table ItemData {
   // whatever this definition is. You'll need to define ItemData in a way that FlatSharp can understand.
   // FlatBuffers is not a serialization format that accepts arbitrary data and types.
   Data:string;
}

Next, add this to the csproj where you installed FlatSharp.compiler:

<ItemGroup>
    <FlatSharpSchema Include="MySchema.fbs" />
</ItemGroup>

This will produce a file with definitions for ItemData and DummyPacket, as well as a serializer for DummyPacket (available under DummyPacket.Serializer).

Now, to add your interfaces. FlatSharp's compiler generates partial classes, so you can simply add this to a file that you maintain:

namespace SomeNamespace
{
    // Add IPacket to the generated definition for DummyPacket.
    // If OpCode does not need to be serialized, then you can just add the read-only property here too.
   public partial class DummyPacket : IPacket { }
}

Take a few minutes and read the FlatSharp samples, as well as the basic documentation from Google's flatbuffers site (both linked on the index page of this repo). I think that will give you some more clarity about pros and cons of FlatBuffers as a choice for serialization format. There are some topics in FlatSharp we haven't touched on here either, such as deserialization mode (lazy, greedy, etc) that the samples will clarify.

The best sample for this is here

It works on non unity project by adding the itemgroup. However, in Unity project, I can't modify the .csproj, any idea to compile the fbs in IDE?

[jamescourtney]

If you're unable to modify the .csproj, you can always invoke the compiler yourself as a prebuild step in whatever way Unity is happy with. With FlatSharp 5, the syntax is:

FlatSharp.Compiler.exe --input SomeFile.fbs --output someDirectory

This produces SomeFile.fbs.generated.cs is the someDirectory subdirectory.