custom types using AmqpContract

[mbroadst]

I see that you can use AmqpContract and AmqpMember in order to make serializable classes to send over a sender link, however, it seems that they only serialize out to arrays/lists of values. Is it possible to serialize a class as a Map instead?

[xinchen10]

[AmqpContract(Name="my.domain:my-class", Encoding=EncodingType.Map)]
class MyClass
{
    // define members here
}

The above will serialize the object as a map with symbol descriptor "my.domain:my-class". Do you need a plain map without descriptor? That cannot be done now but it is possible to add.

[mbroadst]

@xinchen10 I was just reading up on EncodingType :smile: Yes, ideally I would be able to serialize an object as a plain map (map8 or map32), although perhaps I can manually enter the descriptor for that?

[mbroadst]

@xinchen10 also, I'm getting this exception thrown:

System.InvalidCastException: Unable to cast object of type 'System.String' to type 'Amqp.Types.Symbol'.
   at Amqp.Serialization.SerializableType.DescribedMapType.WriteMembers(ByteBuffer buffer, Object container) in d:\Sourc
e\Repos\amqpnetlite-git\src\Serialization\SerializableType.cs:line 798
   at Amqp.Serialization.SerializableType.CollectionType.WriteObject(ByteBuffer buffer, Object graph) in d:\Source\Repos
\amqpnetlite-git\src\Serialization\SerializableType.cs:line 329
   at Amqp.Serialization.AmqpSerializer.WriteObject(AmqpSerializer serializer, ByteBuffer buffer, Object graph) in d:\So
urce\Repos\amqpnetlite-git\src\Serialization\AmqpSerializer.cs:line 128
   at Amqp.Framing.AmqpValue.EncodeValue(ByteBuffer buffer) in d:\Source\Repos\amqpnetlite-git\src\Framing\AmqpValue.cs:
line 67
   at Amqp.Types.Described.Encode(ByteBuffer buffer) in d:\Source\Repos\amqpnetlite-git\src\Types\Described.cs:line 33
   at Amqp.Message.Encode() in d:\Source\Repos\amqpnetlite-git\src\Message.cs:line 140
   at Amqp.SenderLink.Send(Message message, DeliveryState deliveryState, OutcomeCallback callback, Object state) in d:\S
ource\Repos\amqpnetlite-git\src\SenderLink.cs:line 144
   at Amqp.SenderLink.Send(Message message, Int32 millisecondsTimeout) in d:\Source\Repos\amqpnetlite-git\src\SenderLink
.cs:line 100
   at TestSerialization.Program.Main(String[] args) in C:\Users\mbroadst\Development\TestSerialization\TestSerialization
\Program.cs:line 66

when trying to run this piece of code:

[AmqpContract(Name="something:something-else", Encoding = EncodingType.Map)]
public class TestObject
{
    [AmqpMember]
    public string name { get; set; }

    [AmqpMember]
    public string email { get; set; }
}

sender.send(new Message(new TestObject { name="Bob", email="bob@gmail.com" }));

[mbroadst]

@xinchen10 looks like your changes yesterday fix this, and without a need to fix specifying no name. Will close upon release.

[mbroadst]

@xinchen10 oops, I spoke too soon. It works fine when referencing Amqp.Net, however referencing Amqp.Net35 I get the following exception:

PS C:\Users\mbroadst\Development\TestSerialization\TestSerialization\bin\Debug> .\TestSerialization.exe
Amqp.AmqpException: The type 'TestSerialization.Program+TestObject' is not a valid AMQP type and cannot be enc
oded.
   at Amqp.Types.Encoder.WriteObject(ByteBuffer buffer, Object value, Boolean smallEncoding) in C:\Users\mbroadst\Develo
pment\amqpnetlite\src\Types\Encoder.cs:line 375
   at Amqp.Framing.AmqpValue.EncodeValue(ByteBuffer buffer) in C:\Users\mbroadst\Development\amqpnetlite\src\Framing\Amq
pValue.cs:line 81
   at Amqp.Types.Described.Encode(ByteBuffer buffer) in C:\Users\mbroadst\Development\amqpnetlite\src\Types\Described.cs
:line 33
   at Amqp.Message.Encode() in C:\Users\mbroadst\Development\amqpnetlite\src\Message.cs:line 140
   at Amqp.SenderLink.Send(Message message, DeliveryState deliveryState, OutcomeCallback callback, Object state) in C:\U
sers\mbroadst\Development\amqpnetlite\src\SenderLink.cs:line 144
   at Amqp.SenderLink.Send(Message message, Int32 millisecondsTimeout) in C:\Users\mbroadst\Development\amqpnetlite\src\
SenderLink.cs:line 100
   at TestSerialization.Program.Main(String[] args) in C:\Users\mbroadst\Development\TestSerialization\TestSerialization
\Program.cs:line 69

[mbroadst]

@xinchen10 looks like this fixes the issue: https://github.com/Azure/amqpnetlite/pull/22

[mbroadst]

@xinchen10 I have another question which I'm just going to put here since its related. I have a custom RPC type I would like to deserialize off the bus that looks like this:

[AmqpContract(Encoding = EncodingType.Map)]
public class Action
{
    [AmqpMember]
    public string id { get; set; }

    [AmqpMember]
    public string type { get; set; }

    [AmqpMember]
    public string method { get; set; }

    [AmqpMember]
    public string body { get; set; }
}

My issue is with the body member. I would like that member to be any of a number of different deserializable types that I define with AmqpContracts, but would like them to generically come in and process them after the fact based on the type member (a local string lookup => AmqpContract).

The way it's written it fails deserialization, and I think one way to do this is to just have a few Action subclasses and to just iterate through them with try/catch's until one works, but was wondering if its possible to do this more cleanly.

[mbroadst]

it just occurred to me this is exactly what OnDeserializing should be used for :) I just need to figure out how to use the context, I'm not very familiar with this process..

[mbroadst]

To add a little clarity to the question, let's say that I have incoming data that looks like: [id:test, type:TestObject, method:something, body:<data that should be somehow converted during serialization to TestObject>]

[mbroadst]

I might be running into an issue with serialization in general in my attempts to solve this problem. I'm trying to run the follow code:

using System;
using Amqp;
using Amqp.Serialization;

namespace TestSerialization
{
    class Program
    {
        [AmqpContract(Encoding = EncodingType.Map)]
        public class SomeBodyThing
        {
            [AmqpMember]
            public string some { get; set; }
        };

        [AmqpContract(Encoding = EncodingType.Map)]
        public class Action
        {
            [AmqpMember]
            public string id { get; set; }

            [AmqpMember]
            public string action { get; set; }

            [AmqpMember]
            public SomeBodyThing body { get; set; }
        }

        static void Main(string[] args)
        {
            string address = "amqp://192.168.1.9:5672";
            if (args.Length > 0)
            {
                address = args[0];
            }

            try
            {
                Connection connection = new Connection(new Address(address));
                Session session = new Session(connection);

                ReceiverLink receiver = new ReceiverLink(session, "test-serialization", "amq.topic");

                while (true)
                {
                    Message message = receiver.Receive();
                    if (message == null) break;

                    Action command = message.GetBody<Action>();
                    Console.WriteLine(command.id);
                    Console.WriteLine(command.action);
                    Console.WriteLine(command.body);
                }

                receiver.Close();
                session.Close();
                connection.Close();
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
            }
        }
    }
}

And definitely building up a map with QPID's C++ messaging API properly and sending it over the bus. When the C# app receives the message and tries to deserialize it, I get the following exception thrown:

PS C:\Users\mbroadst\Development\TestSerialization\TestSerialization\bin\Debug> .\TestSerialization.exe
Amqp.AmqpException: The format code '209' at frame buffer offset '57' is invalid.
   at Amqp.Serialization.SerializableType.DescribedCompoundType.Initialize(ByteBuffer buffer, Byte formatCode, Int32& si
ze, Int32& count, Int32& encodeWidth, CollectionType& effectiveType) in C:\Users\mbroadst\Development\amqpnetlite-mbroad
st\src\Serialization\SerializableType.cs:line 597
   at Amqp.Serialization.SerializableType.CollectionType.ReadObject(ByteBuffer buffer) in C:\Users\mbroadst\Development\
amqpnetlite-mbroadst\src\Serialization\SerializableType.cs:line 357
   at Amqp.Serialization.AmqpSerializer.ReadObject[T,TAs](AmqpSerializer serializer, ByteBuffer buffer) in C:\Users\mbro
adst\Development\amqpnetlite-mbroadst\src\Serialization\AmqpSerializer.cs:line 135
   at Amqp.Serialization.AmqpSerializer.Deserialize[T](ByteBuffer buffer) in C:\Users\mbroadst\Development\amqpnetlite-m
broadst\src\Serialization\AmqpSerializer.cs:line 66
   at Amqp.Framing.AmqpValue.GetValue[T]() in C:\Users\mbroadst\Development\amqpnetlite-mbroadst\src\Framing\AmqpValue.c
s:line 61
   at Amqp.Message.GetBody[T]() in C:\Users\mbroadst\Development\amqpnetlite-mbroadst\src\Message.cs:line 124
   at TestSerialization.Program.Main(String[] args) in C:\Users\mbroadst\Development\TestSerialization\TestSerialization
\Program.cs:line 101

[xinchen10]

Inter-op on message body is always hard :-) but with AMQP it is at least possible.

Decoding failure How did you build the map object from C++ client? When you apply AmqpContract on the Action class, it has to be a described list/map. Did you set the descriptor from the C++ client? The best practice is to explicitly specific a contract name (the symbol descriptor) otherwise it is defaulted to the class's full name.

When you have a nested custom type, it has to be a described list/map too. Same restrictions apply.

The descriptors are required otherwise it is not possible to support class inheritance.

Solutions to your RPC command encoding. There are two options.

1. You can make body a list and you interpret the objects inside that list based on the value of 'type".
2. Take advantage of the serializer's support for class inheritance and get strongly typed objects. For example, you can have the following class definition. After you get the Action object, you can cast the body to the corresponding parameter type according to the value of 'type'. Again from other clients ensure the correct descriptors are set.

    [AmqpContract(Encoding = EncodingType.Map)]
    [AmqpProvides(typeof(Method1Parameters))]
    [AmqpProvides(typeof(Method2Parameters))]
    public abstract class Parameters
    {
    }

    [AmqpContract(Name="method1-params", Encoding = EncodingType.Map)]
    public class Method1Parameters
    {
        [AmqpMember]
        public int IntParam { get; set; }

        [AmqpMember]
        public string StringParam { get; set; }
    }

    [AmqpContract(Name="method2-params", Encoding = EncodingType.Map)]
    public class Method2Parameters
    {
        [AmqpMember]
        public string StringParam { get; set; }

        [AmqpMember]
        public long LongParam { get; set; }
    }

    [AmqpContract(Name="action", Encoding = EncodingType.Map)]
    public class Action
    {
        [AmqpMember]
        public string id { get; set; }

        [AmqpMember]
        public string type { get; set; }

        [AmqpMember]
        public string method { get; set; }

        [AmqpMember]
        public Parameters body { get; set; }
    }

[mbroadst]

@xinchen10 the map is being built on the c++ side by converting JSON to QPID's Variants (this is the code for the simple command line utility: https://gist.github.com/mbroadst/eb9db3f64a9bd024c413). Alternatively you can run this example code using node-amqp10 if that's preferable for a test: https://gist.github.com/mbroadst/b629bea741e14ea9435e.

I didn't set the descriptor in C++ on the map, if you notice the type coming in is '209' or 'd1' in hex, i.e. a map32 described type. I've been trying to trace through your codec code, but I'm pretty rusty at C# so it's taking me a while. I've actually been working with the amqp1.0 spec for some time at this point, and have never explicitly worked with a situation where the descriptor was set this way for the built-in types provided by the spec - why are they being used? Can't I just refer to this data as a plain map8/map32?

Thanks for the advice on the interop stuff I'll try to grok that while I also try to find out what's going on with the decode failure (I'll never get to the other stuff if this continues to fail).

[xinchen10]

It sounds like the C++ client is sending the object as a plain map without descriptor. On the C# receiver side, you can break after you receive the message and inspect the message.Body property. This is the native AMQP type (a simple type or a described type) decoded from the message body.

[mbroadst]

@xinchen10 yes it is indeed sending a plain map without a descriptor, is it not possible deserialize nested maps? The Deserializer handles a basic map totally fine (without a descriptor, even though the AmqpContract assigns one based on class name), but once it hits the first nested map it throws that exception.

It does seem that if I just extract this way: message.GetBody<Dictionary<string, object>>() it works just fine, but then I lose all the benefit of the AmqpContracts

[xinchen10]

I fixed a bug in encoding custom types as described maps. It happens only to inherited types and the base class has AMQP members. So it should not cause the decoding error you saw.

I think that error is because the C++ client did not add appropriate descriptors for the map object (and the nested one). In AmqpSerializerMapEncodingTest, I added three cases. The last one simulates what you are trying to do. From the C++ client, if you can create a described value with the correct descriptor and the map object as shown in the test case, it should be decoded correctly as a NET object of the custom type.

[mbroadst]

@xinchen10 Thanks for the serialization fix. WRT to the error I was receiving, again if I do a message.GetBody<Dictionary<string, object>>() on the incoming message I get all of the data correctly - this is an exception thrown explicitly when deserializing the data to an AmqpContract. The C++ code is not adding any special descriptors to the map, it's a plain map32 (not a described type) in this case. Is it not possible to use the AmqpContract with incoming nested maps that have no descriptors (this is why it seems like a bug to me, the top level map is also coming in as a map32 with no explicit descriptor)

[xinchen10]

@mbroadst Please try the new release v1.1.2. I believe this scenario should work by using the new SimpleMap encoding type. The PeerToPeer.CustomType sample has been updated with more details. In your case, you would need to define different Action types for different RPC operations and send the concrete type name in message's metadata (properties or application-properties). Without the descriptor, you cannot use AmqpProvides to define known types on the base class and let the serializer auto-resolve the decoding type.

[mbroadst]

@xinchen10 just saw this while driving back from the weekend, and just landed so I'm eager to give it a spin - will let you know! (thanks for all this quick work, it's very much appreciated)

[mbroadst]

seems to work quite well with the SimpleMap encoding, thanks!