Unexpected: Cannot find an implementation class for grain interface -1341526286 with Orleans 3.5.1

[SeppPenner]

I have a strange issue in our outgoing grain call filter implementation after the update to version 3.5.1:

Unexpected: Cannot find an implementation class for grain interface -1341526286
   at Orleans.Runtime.GrainTypeManager.GetTypeInfo(Int32 typeCode, String& grainClass, PlacementStrategy& placement, String genericArguments)
   at Orleans.Runtime.Dispatcher.AddressMessage(Message message)
   at Orleans.Runtime.Dispatcher.AsyncSendMessage(Message message, IGrainContext sendingActivation)
--- End of stack trace from previous location ---
   at Orleans.Runtime.OutgoingCallInvoker.Invoke()
   at IVU.LoRa.GrainInterfaces.GrainCallFilter.OutGrainCallFilter.Invoke(IOutgoingGrainCallContext context) in /builds/ivu/lorawan-orleans/src/IVU.LoRa.GrainInterfaces/GrainCallFilter/OutGrainCallFilter.cs:line 77
   at Orleans.Runtime.OutgoingCallInvoker.Invoke()
   at Orleans.Runtime.GrainReferenceRuntime.InvokeWithFilters(GrainReference reference, InvokeMethodRequest request, InvokeMethodOptions options)
   at Orleans.Runtime.ReminderService.LocalReminderService.LocalReminderData.OnTimerTick()

Was there something changed with the reminders? https://github.com/dotnet/orleans/releases/tag/v3.5.1 doesn't say so.

In line 77 only await context.Invoke(); is called inside a try-catch block.

The code runs on Net6.0, the grain call filter looks like this (Smaller changes done, so the original line 77 isn't line 77 here):

using OpenTracing;
using OpenTracing.Propagation;
using OpenTracing.Tag;
using OpenTracing.Util;
using Orleans;
using Orleans.Runtime;
using Serilog;

namespace MyGrainInterfaces.GrainCallFilter;

public class OutGrainCallFilter : IOutgoingGrainCallFilter
{
    private ITracer tracer;

    public OutGrainCallFilter()
    {
        this.tracer = GlobalTracer.Instance;
    }

    public async Task Invoke(IOutgoingGrainCallContext context)
    {
        if (context.Grain is not GrainReference grain || !grain.GrainIdentity.IdentityString.StartsWith("*grn/MyGrains."))
        {
            try
            {
                await context.Invoke();
            }
            catch (Exception ex)
            {
                Log.Error(ex, "Grain call {@Grain}->{InterfaceMethod}({@Arguments}) failed", context.Grain, context.InterfaceMethod?.Name, context.Arguments);
                throw;
            }
            return;
        }

        // Get tracer
        string grainId;
        var grainIdElements = grain.GrainIdentity.IdentityString.Split('/');
        var grainIdFields = grainIdElements[2].Split('-', '+');

        if (grainIdFields.Length == 2)
        {
            grainId = grainIdFields[0];
        }
        else
        {
            grainId = grainIdFields[1];
        }

        var spanBuilder = tracer
            .BuildSpan($"{grain.GetType().Name}.{grainId}.{context.InterfaceMethod?.Name ?? "---"}")
            .WithTag(Tags.SpanKind, Tags.SpanKindClient);

        using var scope = spanBuilder.StartActive(true);

        // Add tracing information to every call 
        var dictionary = new Dictionary<string, string>();
        scope.Span.SetTag("Filter", "Out");

        tracer.Inject(scope.Span.Context, BuiltinFormats.TextMap, new TextMapInjectAdapter(dictionary));
        RequestContext.Set("TRACE_SPAN_CONTEXT", dictionary);

        try
        {
            await context.Invoke();
        }
        catch (Exception ex)
        {
            Log.Error(ex, "Grain call {GrainName}.{InterfaceMethod}({@Arguments}) failed", grain.GetType().Name, context.InterfaceMethod?.Name, context.Arguments);
            throw;
        }
    }
}

[SeppPenner]

We found the issue here: The reason was that a grain was renamed and an old reminder was stored in the database which wasn't cleaned up.

[benjaminpetit]

Thanks for the update! I will close this issue.