Newtonsoft.Json.JsonSerializationException: Error getting value from 'ScopeId' on 'System.Net.IPAddress'.

[videege]

Hello,

I am using v 1.10.2 of RawRabbit. I have a custom message context that derives from AdvancedMessageContext which stores a list of security claims:

    public class EnhancedMessageContext : AdvancedMessageContext, IEnhancedMessageContext
    {
        public List<SecurityClaim> Claims { get; set; }

        IList<SecurityClaim> IEnhancedMessageContext.Claims
        {
            get { return Claims; }

            set { Claims = value.ToList(); }
        }
    }

   public class SecurityClaim 
   {
       public string Type {get;set;}
       public string Value {get;set;}
   }

I can do Request/Respond operations without error using this custom context, but when I try to do a PublishAsync operation, I get a StackOverflowException. Inspecting this exception made it look like the problem was a reference loop in the object the Json.NET serializer was trying to serialize. It didn't matter what my message was (even a simple one-property class), I was getting a stack overflow. I changed the serializer settings to "ignore" self referencing objects (ReferenceLoopHandling.Ignore) and now I receive the error in the title when the message context is being serialized for publishing:

{Newtonsoft.Json.JsonSerializationException: Error getting value from 'ScopeId' on 'System.Net.IPAddress'. ---> System.Net.Sockets.SocketException: The attempted operation is not supported for the type of object referenced
   at System.Net.IPAddress.get_ScopeId()
   at lambda_method(Closure , Object )
   at Newtonsoft.Json.Serialization.ExpressionValueProvider.GetValue(Object target)
   --- End of inner exception stack trace ---
   at Newtonsoft.Json.Serialization.ExpressionValueProvider.GetValue(Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at RawRabbit.Context.Provider.MessageContextProviderBase`1.SerializeContext(TMessageContext messageContext)
   at RawRabbit.Context.Provider.MessageContextProviderBase`1.GetMessageContext(Guid& globalMessageId)
   at RawRabbit.Operations.Publisher`1.PublishAsync[TMessage](TMessage message, Guid globalMessageId, PublishConfiguration config)
   at RawRabbit.Common.BaseBusClient`1.PublishAsync[T](T message, Guid globalMessageId, Action`1 configuration)

I'm scratching my head a little bit - do you have any insight on why I'm getting this serialization error?

[pardahlman]

@videege - great question. I think that your custom context looks just fine. At first glance, I though you were serializing identity claims, something that does not always succeeds. The exception sounds a lot like this SO post. I think that the problem may occure when your list of claims have items.

For trouble shooting, I would suggest

1. Instantiate a Json Serializer with the same configuration as RawRabbit
2. Create an enhanced context and try to serialize/deserialize the same way as RawRabbit does
3. Repeat 2. but with claims

Report back, and see if you learn anything. It would be helpful if you could attach the json of an message causes this. You could find it my retrieving a message in the management plugin.

[videege]

I tried your suggested steps above. Whether or not the list of claims was populated made no difference, I still got a StackOverflowException.

I then started trying to serialize just a plain AdvancedMessageContext:

//_serializer is a JsonSerializer injected using the standard configuration provided by RawRabbit

var ctx = new AdvancedMessageContext()
{
    GlobalRequestId = Guid.NewGuid(),
    Nack = () => { Console.WriteLine("hi"); },
    RetryInfo = new RetryInformation() { NumberOfRetries = 0, OriginalSent = DateTime.Now },
    RetryLater = (ts) => { Console.WriteLine(ts); }
};
var serializer = new JsonMessageSerializer(_serializer, cfg =>
{
    cfg.ReferenceLoopHandling = ReferenceLoopHandling.Error;
});
var serializedContext = serializer.Serialize(ctx);

If I don't change the ReferenceLoopHandling property to Error, I get a stack overflow. Changing it to error reveals a bunch of difficulty in serializing the Action properties of the AdvancedMessageContext:

"Self referencing loop detected for property 'module' with type 'System.Reflection.RuntimeModule'. Path 'nack.method.module.assembly.entryPoint'."

If I set the ReferenceLoopHandling property to Ignore, I can serialize the context, but it includes a large amount of JSON for the Nack and RetryLater properties:

"{\"$id\":\"1\",\"$type\":\"RawRabbit.Context.AdvancedMessageContext, RawRabbit\",\"nack\":{\"$id\":\"2\",\"$type\":\"System.Action, System.Private.CoreLib\",\"method\":{\"$type\":\"System.Reflection.RuntimeMethodInfo, System.Private.CoreLib\",\"name\":\"<.ctor>b__3_0\",\"declaringType\":\"RawRabbit.AspNet.Sample.Controllers.ValuesController+<>c, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"reflectedType\":\"RawRabbit.AspNet.Sample.Controllers.ValuesController+<>c, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"memberType\":8,\"metadataToken\":100663314,\"module\":{\"$type\":\"System.Reflection.RuntimeModule, System.Private.CoreLib\",\"mdStreamVersion\":131072,\"fullyQualifiedName\":\"D:\\\\Playground\\\\RawRabbit\\\\sample\\\\RawRabbit.AspNet.Sample\\\\bin\\\\Debug\\\\netcoreapp1.0\\\\RawRabbit.AspNet.Sample.dll\",\"moduleVersionId\":\"96f1e7ca-30ea-4e54-9eeb-bbe4458441f5\",\"metadataToken\":1,\"scopeName\":\"RawRabbit.AspNet.Sample.dll\",\"name\":\"RawRabbit.AspNet.Sample.dll\",\"assembly\":{\"$type\":\"System.Reflection.RuntimeAssembly, System.Private.CoreLib\",\"codeBase\":\"file:///D:/Playground/RawRabbit/sample/RawRabbit.AspNet.Sample/bin/Debug/netcoreapp1.0/RawRabbit.AspNet.Sample.dll\",\"fullName\":\"RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"entryPoint\":{\"$type\":\"System.Reflection.RuntimeMethodInfo, System.Private.CoreLib\",\"name\":\"Main\",\"declaringType\":\"RawRabbit.AspNet.Sample.Program, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"reflectedType\":\"RawRabbit.AspNet.Sample.Program, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"memberType\":8,\"metadataToken\":100663297,\"methodHandle\":{\"$type\":\"System.RuntimeMethodHandle, System.Private.CoreLib\",\"value\":{\"$type\":\"System.IntPtr, System.Private.CoreLib\"}},\"attributes\":150,\"callingConvention\":1,\"returnType\":\"System.Void, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"returnTypeCustomAttributes\":{\"$type\":\"System.Reflection.RuntimeParameterInfo, System.Private.CoreLib\",\"parameterType\":\"System.Void, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"hasDefaultValue\":true,\"metadataToken\":134217728,\"position\":-1,\"customAttributes\":{\"$type\":\"System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Reflection.CustomAttributeData, System.Private.CoreLib]], System.Private.CoreLib\",\"$values\":[]}},\"returnParameter\":{\"$type\":\"System.Reflection.RuntimeParameterInfo, System.Private.CoreLib\",\"parameterType\":\"System.Void, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"hasDefaultValue\":true,\"metadataToken\":134217728,\"position\":-1,\"customAttributes\":{\"$type\":\"System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Reflection.CustomAttributeData, System.Private.CoreLib]], System.Private.CoreLib\",\"$values\":[]}},\"isPublic\":true,\"isStatic\":true,\"isHideBySig\":true,\"customAttributes\":{\"$type\":\"System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Reflection.CustomAttributeData, System.Private.CoreLib]], System.Private.CoreLib\",\"$values\":[]}},\"definedTypes\":{\"$type\":\"System.RuntimeType[], System.Private.CoreLib\",\"$values\":[\"RawRabbit.AspNet.Sample.Program, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Startup, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Controllers.ValuesController, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Startup+<>c, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Controllers.ValuesController+<>c, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Controllers.ValuesController+<>c__DisplayClass4_0, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\"]},\"location\":\"D:\\\\Playground\\\\RawRabbit\\\\sample\\\\RawRabbit.AspNet.Sample\\\\bin\\\\Debug\\\\netcoreapp1.0\\\\RawRabbit.AspNet.Sample.dll\",\"imageRuntimeVersion\":\"v4.0.30319\",\"exportedTypes\":{\"$type\":\"System.Type[], System.Private.CoreLib\",\"$values\":[\"RawRabbit.AspNet.Sample.Program, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Startup, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\",\"RawRabbit.AspNet.Sample.Controllers.ValuesController, RawRabbit.AspNet.Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\"]},\"customAttributes\":{\"$type\":\"System.Collections.ObjectModel.ReadOnlyCollection`1[[System.Reflection.CustomAttributeData, System.Private.CoreLib]], System.Private.CoreLib\",\"$values\":[{\"$id\":\"3\",\"$type\":\"System.Reflection.CustomAttributeData, System.Private.CoreLib\",\"attributeType\":\"System.Runtime.CompilerServices.CompilationRelaxationsAttribute, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"constructor\":{\"$type\":\"System.Reflection.RuntimeConstructorInfo, System.Private.CoreLib\",\"name\":\".ctor\",\"memberType\":1,\"declaringType\":\"System.Runtime.CompilerServices.CompilationRelaxationsAttribute, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"reflectedType\":\"System.Runtime.CompilerServices.CompilationRelaxationsAttribute, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e\",\"metadataToken\":100664987,\"module\":{\"$type\":\"System.Reflection.RuntimeModule, System.Private.CoreLib\",\"mdStreamVersion\":131072,\"fullyQualifiedName\":\"C:\\\\Program 
//continues on....

It seems like these Action type properties shouldn't be serialized, or there is some issue with the serialization settings. Is the issue possibly related to how I am registering RawRabbit with the DI system? Using the Sample project, I am doing this:

            services
                .AddRawRabbit<AdvancedMessageContext>(
                    cfg => cfg.SetBasePath(_rootPath).AddJsonFile("rawrabbit.json"),
                    ioc => ioc
                        .AddSingleton(LoggingFactory.ApplicationLogger))
                .AddSingleton<IConfigurationEvaluator, AttributeConfigEvaluator>()
                .AddMvc();

[videege]

OK, I think I found a lead on the issue. I only encounter this serialization issue when publishing a message in the process of responding to another message, like this:

var client = provider.GetService<IBusClient<EnhancedMessageContext>>();
client.RespondAsync<RequestMessage, ResponseMessage>(async (msg, context) => {
    //do some stuff, then I want to publish an event:
   await client.PublishAsync(new ThingHappened(), context.GlobalRequestId); //stack overflow (or serialization error if I change the serialization settings)
});

I think what's happening is I get the first message, the ContextEnhancer comes through and sets up the Nack and RetryLater portions of the context. Then when I try to publish a message, I try to correlate the new message with the existing message. This seems to do a lookup on the context object in the ContextProvider's internal dictionary - it finds the EnhancedMessageContext, pulls it out, and tries to serialize it. This blows up because the Nack and RetryLater actions cannot be serialized.

To summarize:

//inside a message handler with an AdvancedMessageContext
await client.PublishAsync(new ThingHappened(), context.GlobalRequestId); //fails
await client.PublishAsync(new ThingHappened()); //fails (uses an ambiently stored global request ID?)
await client.PublishAsync(new ThingHappened(), Guid.NewGuid()); // succeeds

What's the best way to work around this?

[pardahlman]

Of course! The problem occurs when trying to forward a message context that contains func that in turn does other things. I'm stream-lining the message context management for 2.0 at this very moment, and for that version this wont be a problem.

There is a quick fix for this problem in 1.x, that should work (even though it is not the most beautiful solution). The context is retrieved in the Message Context Provider. Before serializing it, you could check if it is an advanced message context, and in that case null the properties

var advanced = context as IAdvancedMessageContext;
if (advanced != null)
{
    advanced.Nack = null;
    advanced.RetryLater = null;
}

My suggestion is that you implement your own IMessageContextProvider, based on the default one but with the snippet above in the serialization method. Looking at the base class I see that the methods are not marked as virtual, so I guess your best option is copy/paste (sorry!).

[pardahlman]

I've created a separate issue (#137) to make sure that relevant methods are marked as virtual in 2.0.

[videege]

Thanks - I took your suggestion and implemented an IMessageContextProvider.