Rebus deserialization issue

DuleDimitrov123 commented 5 months ago

Hi,

I am using dotnet CAP library for publishing events (because I want outbox pattern) and Rebus for reading events from RabbitMQ (works perfectly with CAP and RabbitMQ.Client libraries).

The problem is with serialization of the message in Rebus.

When I don't explicitly configure serializer (looks like System.Text.Json is default) the exceptions ocures that returnType (for serialization) is null (detailed exception in my first comment).

When I use NewtonsoftJson exception about not dispatching to any handlers (detailed exception in my first comment).

When I specify my explicit serializer it works, but I would like to avoid using it since I would like other system to be able to use my event without explicit definition of serializer (detailed exception and setup in my first comment).

Tried with couple of versions, same problem, but curently using Rebus with version 8.2.2 and dotnet 8.

My configuration:

...(other nonRebus setup)
 services.AddRebus(configure => configure
    .Transport(c =>
    {
        c.UseRabbitMq(rabbitMqConfig.ConnectionString, rabbitMqConfig.QueueName)
            .InputQueueOptions(o =>
            {
                o.SetAutoDelete(false);
                o.AddArgument("x-message-ttl", rabbitMqConfig.TimeToLive);
            });
    })
        //.Serialization(s => s.Register(c => new CustomJsonSerializer())));
        //.Serialization(s => s.UseNewtonsoftJson()));

services.AutoRegisterHandlersFromAssemblyOf<MyEventHandler>();

var bus = services.BuildServiceProvider().GetRequiredService<IBus>();
await bus.Subscribe<MyEvent>();
...(other nonRebus setup)

and my (testing) handler:

public class MyEventHandler : IHandleMessages<MyEvent>
{
    public Task Handle(MyEvent message)
    {
        Console.WriteLine("IT WORKS");

        return Task.CompletedTask;
    }
}

DuleDimitrov123 commented 5 months ago

Detailed exceptions and setup for 3 cases:

without explicit serializer (looks like System.Text.Json is by default) I receive in error queue the following exception:


5 unhandled exceptions: 2/5/2024 11:43:03 AM +01:00: System.FormatException: Could not deserialize JSON text: '{"Id":34282,"CollabInfoId":132,"Application":"application","CompanyId":"d6b4f3bc-0f15-46a1-af86-b3dd369158c4","Name":"Name","Extension":"Extension","SizeQuantity":123,"Path":"path","Type":"Type","CreatedBy":"aecdbdc0-8cbc-48e7-a97c-6d91c1b01a6c","DateCreated":"0001-01-01T00:00:00","DateModified":"0001-01-01T00:00:00","FailedIterationsCounter":0,"MaxIterationsNumber":0}'
---> System.ArgumentNullException: Value cannot be null. (Parameter 'returnType')
at System.Text.Json.ThrowHelper.ThrowArgumentNullException(String parameterName)
at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(String bodyString, Type type)
--- End of inner exception stack trace ---
at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(String bodyString, Type type)
at Rebus.Serialization.Json.SystemTextJsonSerializer.GetMessage(TransportMessage transportMessage, Encoding bodyEncoding)
at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(TransportMessage transportMessage)
at Rebus.Compression.UnzippingSerializerDecorator.Deserialize(TransportMessage transportMessage)
at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func`1 next)
at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func`1 next)

2/5/2024 11:43:03 AM +01:00: System.FormatException: Could not deserialize JSON text: '{"Id":34282,"CollabInfoId":132,"Application":"application","CompanyId":"d6b4f3bc-0f15-46a1-af86-b3dd369158c4","Name":"Name","Extension":"Extension","SizeQuantity":123,"Path":"path","Type":"Type","CreatedBy":"aecdbdc0-8cbc-48e7-a97c-6d91c1b01a6c","DateCreated":"0001-01-01T00:00:00","DateModified":"0001-01-01T00:00:00","FailedIterationsCounter":0,"MaxIterationsNumber":0}' ---> System.ArgumentNullException: Value cannot be null. (Parameter 'returnType') at System.Text.Json.ThrowHelper.ThrowArgumentNullException(String parameterName) at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options) at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(String bodyString, Type type) --- End of inner exception stack trace --- at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(String bodyString, Type type) at Rebus.Serialization.Json.SystemTextJsonSerializer.GetMessage(TransportMessage transportMessage, Encoding bodyEncoding) at Rebus.Serialization.Json.SystemTextJsonSerializer.Deserialize(TransportMessage transportMessage) at Rebus.Compression.UnzippingSerializerDecorator.Deserialize(TransportMessage transportMessage) at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func1 next) at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func1 next) at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func1 next) at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func1 next)


2. with .Serialization(s => s.UseNewtonsoftJson()));

1 unhandled exceptions: 2/5/2024 11:46:54 AM +01:00: Rebus.Exceptions.MessageCouldNotBeDispatchedToAnyHandlersException: Message with ID knuth-13090383562881270373 and type Newtonsoft.Json.Linq.JObject, Newtonsoft.Json could not be dispatched to any handlers (and will not be retried under the default fail-fast settings) at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.Process(IncomingStepContext context, Func1 next) at Rebus.Sagas.LoadSagaDataStep.Process(IncomingStepContext context, Func1 next) at Rebus.Pipeline.Receive.ActivateHandlersStep.Process(IncomingStepContext context, Func1 next) at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.Process(IncomingStepContext context, Func1 next) at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.Process(IncomingStepContext context, Func1 next) at Rebus.DataBus.ClaimCheck.HydrateIncomingMessageStep.Process(IncomingStepContext context, Func1 next) at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.Process(IncomingStepContext context, Func1 next) at Rebus.Retry.Simple.DefaultRetryStep.Process(IncomingStepContext context, Func1 next)


3. with following code for explicit serializer (.Serialization(s => s.Register(c => new CustomJsonSerializer())));), it works, but I don't want to hardcode serialization process

public class CustomJsonSerializer : ISerializer
{
    private readonly JsonSerializerOptions _jsonSerializerOptions;

    public CustomJsonSerializer()
    {
        _jsonSerializerOptions = new JsonSerializerOptions
        {
            PropertyNameCaseInsensitive = true,
            PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
        };
    }

    public Task<Message> Deserialize(TransportMessage transportMessage)
    {
        if (transportMessage == null)
            throw new ArgumentNullException(nameof(transportMessage));

        var jsonString = Encoding.UTF8.GetString(transportMessage.Body);

        //works with and without serialization options
        var body = JsonSerializer.Deserialize(jsonString, typeof(MyEvent), _jsonSerializerOptions);
        //var body = JsonSerializer.Deserialize(jsonString, typeof(MyEvent));

        var message = new Message(transportMessage.Headers, body);

        return Task.FromResult(message);
    }

    public Task<TransportMessage> Serialize(Message message)
    {
        throw new NotImplementedException();
    }
}



I have tried different serialization methods. System.Text.Json (provided exception), Newtonsoft (provided exception) and custom serializer, but I would like to avoid custom serializers since I would like other system to be able to use my event without explicit definition of serializer. Any advice?

mookid8000 commented 4 months ago

Rebus' built-in JSON serializers both need the rbs2-msg-type header to contain a value that allows the receiver to look up the .NET type and pass it as a parameter when deserializing.

From the error you're seeing, it looks like the type lookup returned NULL, which most likely means that the receiver does not have access to the .NET type mentioned in the header.

Please ensure that the rbs2-msg-type header has a value that gives a proper result when Type.GetType(value) gets called, or implement an alternative strategy by registering your own IMessageTypeNameConvention.

One way of easily customizing the type names is to use Rebus' built-in type name customizer, which can e.g. be used like this:

services.AddRebus(
    configure => configure
        (...)
        .Serialization(s => s.UseCustomMessageTypeNames()
            .AddWithShortNames(new[] { typeof(string) }))
);

(in this case using short class names like "MyEvent" instead of the default "short, assembly-qualified type names")

rebus-org / Rebus.RabbitMq

Rebus deserialization issue #116