dotnet / orleans

Cloud Native application framework for .NET
https://docs.microsoft.com/dotnet/orleans
MIT License
10.06k stars 2.03k forks source link

Errors from SiloHost #4811

Closed srinathgnath closed 2 years ago

srinathgnath commented 6 years ago

Hello,

I have a Silohost & client running on local machine and connects through consul membership. Everything works fine, but I was able to find below exceptions. What are these excepions? Do I need to fix something before these code move to higher environments?

{"Timestamp":"2018-08-01T17:00:58.0013497-05:00","Level":"Error","MessageTemplate":"Worker pool thread Runtime.Scheduler.WorkerPoolThread/3 (ManagedThreadId=40) has been busy for long time: Executing Task Id=97879 Status=Running for 00:00:59.7464953 on WorkItem=WorkItemGroup:Name=[Activation: S10.XXX.XX.XXX:XX:270856538*grn/9B57CB77/00000000+XXXXZ505@01be8caf

GrainType=XXXXX.XXXXX.Grains.CommandBus Placement=ActivationCountBasedPlacement],WorkGroupStatus=Running Executing for 00:00:59.7555196. WorkItemGroup Details:

WorkItemGroup:Name=[Activation: S10.XX.XX.XX:XX:270856538grn/9B57CB77/00000000+17543CZ505@01be8caf #GrainType=XXXX.XXXXX.Grains.CommandBus Placement=ActivationCountBasedPlacement], WorkGroupStatus=Running. Currently QueuedWorkItems=0; Total EnQueued=2; Total processed=1; Quantum expirations=0; TaskRunner=ActivationTaskScheduler-14:Queued=0; Detailed SchedulingContext=<[Activation: S10.XXX.XX.XXX:XXX:270856538grn/9B57CB77/0000000000000000000000000000000006ffffff9b57cb77+17543CZ505-0x1E0EA515@01be8caf #GrainType=XXXX.XXXXX.Grains.CommandBus Placement=ActivationCountBasedPlacement State=Valid NonReentrancyQueueSize=0 EnqueuedOnDispatcher=0 InFlightCount=1 NumRunning=1 IdlenessTimeSpan=736906.22:00:58.0003454 CollectionAgeLimit=02:00:00 CurrentlyExecuting=NewPlacement Request XXX.XXX.XX.XX:XXX:270856538cli/a64bf6b3@bf402085->S10.XXX.XX.XXX:XX:270856538grn/9B57CB77/00000000+17543CZ505@01be8caf

5: ]>.","Properties":{"EventId":{"Id":101212},"SourceContext":"Runtime.Scheduler.WorkerPoolThread/3","Orleans":"XXXXXX","EnvironmentUserName":"XXXXX\XXXXXX","MachineName":"XXXXXXX",

"ProcessId":15836,"ProcessName":"XXXX.XXXXX.Host","ThreadId":40}}

{"Timestamp":"2018-08-01T17:00:59.7092269-05:00","Level":"Error","MessageTemplate":"Watchdog Freeze Alert: Orleans.Runtime.asynTask.SafeTimerBase.Membership.ProbeTimer did not fire on time. Last fired at 2018-08-01 22:00:42.396 GMT, 00:00:17.3124134 since previous fire, should have fired after 00:00:10.","Properties":{"EventId":{"Id":101412},"SourceContext": "Orleans.Runtime.GrainTimer","Orleans":"SiloHostBuilder","EnvironmentUserName":"XXXX\XXXXX","MachineName":"XXXXXX-XXX","ProcessId":15836,"ProcessName":"XXXXX.XXXX.Host","ThreadId":40}}

{"Timestamp":"2018-08-01T17:01:03.2601117-05:00","Level":"Error","MessageTemplate":"Watchdog Freeze Alert: Orleans.Runtime.asynTask.SafeTimerBase.Membership.IAmAliveTimer did not fire on time. Last fired at 2018-08-01 21:55:44.998 GMT, 00:05:18.2619388 since previous fire, should have fired after 00:05:00.","Properties":{"EventId":{"Id":101412}, "SourceContext":"Orleans.Runtime.GrainTimer","Orleans":"SiloHostBuilder","EnvironmentUserName":"XXXX\XXXXXX","MachineName":"XXXXXX-XXX","ProcessId":15836," ProcessName":"XXXXX.XXXXX.Host","ThreadId":40}}

jason-bragg commented 6 years ago

What version of Orleans?

srinathgnath commented 6 years ago

Orleans 2.0.3

jason-bragg commented 6 years ago

This looks like the service is taking unexpectedly long periods of time to perform some actions? This could be caused by locks, load, GC hits... ? Is this occurring under load?

ReubenBond commented 6 years ago

.NET Core or .NET Framework? If you upgrade to 2.0.4, do you still see the issues?

ghost commented 2 years ago

We are marking this issue as stale due to the lack of activity in the past six months. If there is no further activity within two weeks, this issue will be closed. You can always create a new issue based on the guidelines provided in our pinned announcement.

ghost commented 2 years ago

This issue has been marked stale for the past 30 and is being closed due to lack of activity.