Sometimes Unsubscribe method hangs

[GraySerg]

Hi. Sometimes Unsubscribe method hangs, it looks like deadlock. It occures only on production system with high load, i can't reproduce it locally. I have a lot of internal classes that subscribe on start of windows service, and unsubscribe on stop. And on every service stopping random class hands on unsubscribe. My class looks like

    internal abstract class EasyMonitorService: IMonitorService
    {
        readonly protected Easy.MessageHub.MessageHub hub;
        readonly List<System.Guid> subscriptions;
        public bool IsStarted { get; private set; }
        public EasyMonitorService(Easy.MessageHub.MessageHub hub)
        {
            this.hub = hub;
            subscriptions = new List<Guid>(1); 
        }
        public void Start()
        {
            if (!IsStarted)
            {
                subscriptions.AddRange(OnStart());
                IsStarted = true;
            }
        }
        protected abstract IEnumerable<Guid> OnStart();
        public void Stop()
        {
            foreach (var subscribeGuid in subscriptions)
            {
                hub.Unsubscribe(subscribeGuid);
            }
            OnStop();
        }
        protected virtual void OnStop()
        {

        }
    }

[NimaAra]

This library does not utilize multiple threads. All operations are executed on whatever thread your application is doing the publication/subscription on.

Were there any exceptions thrown by the hub? How about any exceptions thrown by the application?

I cannot provide much help without knowing about your application as a whole. I suggest capturing a dump when this happens in production and going through it to see what the thread was doing at the time.

Alternatively you can download the source, add logging to Subscriptions.Unregister and observe the behaviour.

[GraySerg]

No exceptions, just hangs. Start and stop occures in one thread. It's a pity, but impossible to take dump from production server.

[NimaAra]

Try referencing the library directly and adding some logging. I cannot think of anything else.

[NimaAra]

I am closing the issue for now, feel free to re-open it once you have tried my suggestion or have more data we can work with.

[ummarbhutta]

Hi @NimaAra and @GraySerg

I think that I hit a similar issue on one of our test server. The state of application is totally non responsive state, and interestingly all other threads of application stopped as well. Bit details as follows

Environment

• EasyMessageHub version 4.0.1, I just noticed latest is 5.0.0 is it too old?
• ASP dot net core 3.1 based WebAPI application.
• Deployed on Ubuntu 18.04 LTS.
• In a Docker container.
• docker version is Docker version 19.03.8, build afacb8b7f0
• docker-compose version is docker-compose version 1.25.5, build 8a1c60f6

Observation

• In our application we are making ~3 subscriptions in a minutes and then unsubscribing them at end.
• We left our system running for overnight testing, and the second day in morning the whole application was stuck and the RESTFul endpoints are not accessible as well.
• Reviewing the log reveals that it went for MessageHub.Instance.Unsubscribe and never returns, blocking everything.
• Since then I have been retrying to reproduce it again, but it haven't appeared again.

In short, I think what @GraySerg has reported, seems to be a real bug, I think we should try to find the root cause of issue.

@GraySerg , did you find any clue on your side?

[NimaAra]

@ummarbhutta Can you please upgrade to the latest and let me know if this happen again.