microsoft / service-fabric

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale.
https://docs.microsoft.com/en-us/azure/service-fabric/
MIT License
3.03k stars 399 forks source link

Service Fabric - Errors/Warnings #795

Open nareshkhatri81 opened 6 years ago

nareshkhatri81 commented 6 years ago

Team,

we see couple of warnings/errors in logs. Can someone try to explain us what could be root cause ?

The description for Event ID 326 from source ESE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Fabric 8504 (00000000-0000-0000-0000-000000001001:131671378213481414): 5 C:\ProgramData\SF\vm1\Fabric\work\NS\P_00000000-0000-0000-0000-000000001001\R_131671378213481414\NS.edb 0 [1] 0.000, [2] 0.000, [3] 0.032, [4] 0.000, [5] 0.000, [6] 0.015, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000, [11] 0.000, [12] 0.000. 1 0

alexwun commented 6 years ago

What are the warnings and errors from the Service Fabric source in Event Viewer? Or health events showing up in Azure portal for this partition (if any)?

nareshkhatri81 commented 6 years ago

@alexwun - i don't see any service fabric source in event viewer. does it come under applications,system etc ? does SF logs all errors in event viewer ? we are running cluster onpremise. is there any tool you suggest for looking into all errors in cluster ?

alexwun commented 6 years ago

Local Service Fabric error and warning logs should appear in Event Viewer under "Custom Views" -> "Administrative Events" by default. That said, not all local warnings necessarily indicate an issue and are cause for concern. It's better to look at the health of the overall cluster through the Azure portal first to determine if there are any real issues:

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-view-entities-aggregated-health

nareshkhatri81 commented 6 years ago

@alexwun i can see logs in event viewer but when i try to query same using powershell Get-EventLog. its not able to find logName with that name. am i missing something ?

Get-EventLog -LogName 'Microsoft-Service Fabric/Admin' -Source 'Microsoft-Service Fabric' -EntryType Error -After (Get-Date).AddDays(-2) | Sort-Object Time

is it possible to log verbose information in event viewer ? or we have subscribe to channels and send to some anlaytics framwork.

we are running cluster on premise not in cloud.

Thanks, Naresh Khatri

alexwun commented 6 years ago

Service Fabric Explorer can still be used in on-premise clusters. You just need to use the HTTP port configured in the cluster manifest:

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-visualizing-your-cluster

There are also PowerShell cmdlets for looking at the aggregated health of the cluster and applications:

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-understand-and-troubleshoot-with-system-health-reports

Start with those if you're trying to add telemetry, not the Event Viewer logs.

JohnNilsson commented 6 years ago

I have a node in an azure cluster with this same issue (ESE events not understood by event viewer) On a node without the issue I can find the register key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Application\ESE on the node with this issue that key is missing. So I guess something in the runtime install/upgrade has failed.

Is there a way to trigger a repair of the service fabric installation?