search

Azure / Kusto-Lightingest

Kusto Lightingest tool
MIT License
2 stars 3 forks source link

[Bug] LightIngestLinux failing with error "Unable to load shared library 'kernel32.dll' or one of its dependencies." #4

Closed tyagi closed 8 months ago

tyagi commented 9 months ago

Hi team,

LightIngestLinux version of the tool is failing with error -

$$ALERT[Crash_UnrecoverableException]: +++ Crash! CrashScenario=UnrecoverableException
Crashing due to an unrecoverable exception of type System.TypeInitializationException: 
+++ Call stack:
   at Kusto.Cloud.Platform.Utils.CrashTrail..ctor(CrashScenario scenario, String message, Exception ex)
   at Kusto.Cloud.Platform.Utils.CrashHandler.CrashImpl(CrashScenario scenario, String message, Exception ex, Action`1 failAction)
   at Kusto.Cloud.Platform.Utils.CrashHandler.Crash(CrashScenario scenario, String message)
   at Kusto.Cloud.Platform.Utils.CrashHandler.OnFirstChanceException(Object sender, FirstChanceExceptionEventArgs e)
   at Kusto.Cloud.Platform.Utils.NativeThread.SetThreadDescription(String threadDescription)
   at Kusto.Cloud.Platform.Utils.AsyncInvoker.AsyncInvokerCapsuleAction`1.LongRunningThread()
   at System.Threading.Thread.StartCallback()

Fatal error. Internal CLR error. (0x80131506)
   at Kusto.Cloud.Platform.Utils.NativeProcess.GetCurrentProcess()
   at Kusto.Cloud.Platform.Utils.ExtendedEnvironment.FailFast(System.String, System.Exception)
   at Kusto.Cloud.Platform.Utils.CrashHandler.CrashImpl(Kusto.Cloud.Platform.Utils.CrashScenario, System.String, System.Exception, System.Action`1<System.String>)
   at Kusto.Cloud.Platform.Utils.CrashHandler.Crash(Kusto.Cloud.Platform.Utils.CrashScenario, System.String)
   at Kusto.Cloud.Platform.Utils.CrashHandler.OnFirstChanceException(System.Object, System.Runtime.ExceptionServices.FirstChanceExceptionEventArgs)
   at Kusto.Cloud.Platform.Utils.NativeThread.SetThreadDescription(System.String)
   at Kusto.Cloud.Platform.Utils.AsyncInvoker+AsyncInvokerCapsuleAction`1[[System.__Canon, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].LongRunningThread()
   at System.Threading.Thread.StartCallback()
Aborted

Traces

Filename: 9923-20231221070206-CPC-ankur-MYBL8-LightIngestLinux.1-94598-0a14eedfbbc5415789931394d4e1a1b8.csv

,2023-12-21T07:02:06.0402136Z,,Information,CPC-ankur-MYBL8,N/A,94598,1,,,,,,"Hey, cool, you're reading traces written by RollingCsvTraceListener2! Name=RollingCsvTraceListener2[CreateAndInitialize], NumRollingWriteStreams=1"
LightIngest,2023-12-21T07:02:06.0484426Z,,Information,CPC-ankur-MYBL8,N/A,94598,1,,,,,,"Hey, cool, you're reading traces written by RollingCsvTraceListener2! (with with an appropriate context) Name=RollingCsvTraceListener2[CreateAndInitialize], NumRollingWriteStreams=1"
LightIngest,2023-12-21T07:02:07.6773077Z,P.ConsumerPool,Warning,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,C258A440,"AtomicCounter.Resize: Capacity was changed for resource: 'AuthenticationMetadataByClusterUrl' from '0' to '64'"
LightIngest,2023-12-21T07:02:07.6800610Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,9EAF0EFA,"AuthenticationMetadataProvider.GetAadAuthenticationMetadataForCluster: global authentication metadata is not set. Will try to obtain metadata from cluster."
LightIngest,2023-12-21T07:02:07.6887976Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,a198530d-87a6-46f8-aa48-e4fa34a0410b,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,698837D2,"AuthenticationMetadataProvider.ExternalGetAuthenticationMetadataForClusterAsync: Called with ClusterUrl = '<Redacted>/'"
LightIngest,2023-12-21T07:02:08.4918226Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94640,a198530d-87a6-46f8-aa48-e4fa34a0410b,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,F51A6BAF,"AuthenticationMetadataProvider.ExternalGetAuthenticationMetadataForClusterAsync: Completed for ClusterUrl = '<Redacted>/'"
LightIngest,2023-12-21T07:02:08.4953351Z,P.MonitoredActivity,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94640,a198530d-87a6-46f8-aa48-e4fa34a0410b,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,27A39996,"MonitoredActivityCompletedSuccessfully: getContext=[null], key=<Redacted>/, ActivityType=P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl, Timestamp=2023-12-21T07:02:07.6864906Z, ParentActivityId=731d9eee-348e-4d9d-8f32-7b88c6a369a2, Duration=807.8685 [ms], HowEnded=Success"
LightIngest,2023-12-21T07:02:08.5190466Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,9EAF0EFA,"AuthenticationMetadataProvider.GetAadAuthenticationMetadataForCluster: global authentication metadata is not set. Will try to obtain metadata from cluster."
LightIngest,2023-12-21T07:02:08.5192147Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,170fc47c-1752-47a2-b4a4-147a4459f9ec,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,698837D2,"AuthenticationMetadataProvider.ExternalGetAuthenticationMetadataForClusterAsync: Called with ClusterUrl = '<Redacted>'"
LightIngest,2023-12-21T07:02:08.6028212Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94632,170fc47c-1752-47a2-b4a4-147a4459f9ec,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,F51A6BAF,"AuthenticationMetadataProvider.ExternalGetAuthenticationMetadataForClusterAsync: Completed for ClusterUrl = '<Redacted>'"
LightIngest,2023-12-21T07:02:08.6030272Z,P.MonitoredActivity,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94632,170fc47c-1752-47a2-b4a4-147a4459f9ec,731d9eee-348e-4d9d-8f32-7b88c6a369a2,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,27A39996,"MonitoredActivityCompletedSuccessfully: getContext=[null], key=<Redacted>, ActivityType=P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl, Timestamp=2023-12-21T07:02:08.5191755Z, ParentActivityId=731d9eee-348e-4d9d-8f32-7b88c6a369a2, Duration=83.8217 [ms], HowEnded=Success"
LightIngest,2023-12-21T07:02:08.6037268Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,BA377BBF,"AuthenticationMetadataProvider.GetDstsAuthenticationMetadataForCluster: global authentication metadata is not set. Will try to obtain metadata from cluster."
LightIngest,2023-12-21T07:02:09.1057708Z,KD.RestClient2,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94640,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,A3ACC93B,"$$HTTPREQUEST[RestClient2]: Verb=POST, Uri=<Redacted>/v1/rest/mgmt, App=LightIngestLinux.1, User=CPC-ankur-MYBL8\ankurtyagi, ClientVersion=Runtime:{.NET_6.0.10/CLRv6.0.10/6.0.10}, ClientRequestId=KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c"
LightIngest,2023-12-21T07:02:09.4570747Z,KD.RestClient2,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94639,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,21EBEC05,"$$HTTPREQUEST_RESPONSEHEADERRECEIVED[RestClient2]: Verb=POST, Uri=<Redacted>/v1/rest/mgmt, App=LightIngestLinux.1, User=CPC-ankur-MYBL8\ankurtyagi, ClientVersion=Runtime:{.NET_6.0.10/CLRv6.0.10/6.0.10}, StatusCode=OK"
LightIngest,2023-12-21T07:02:09.4629966Z,P.MonitoredActivity,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94639,731d9eee-348e-4d9d-8f32-7b88c6a369a2,731d9eee-348e-4d9d-8f32-7b88c6a369a2,KD.RestClient.ExecuteControlCommand,KD2RunCommand;3b2422d1-f28d-4825-a6de-86619962518c,27A39996,"MonitoredActivityCompletedSuccessfully: ActivityType=KD.RestClient.ExecuteControlCommand, Timestamp=2023-12-21T07:02:07.6460399Z, ParentActivityId=731d9eee-348e-4d9d-8f32-7b88c6a369a2, Duration=1816.925 [ms], HowEnded=Success"
LightIngest,2023-12-21T07:02:09.5104519Z,KD.RestClient2,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,KD.RestClient.ExecuteControlCommand,KD2RunCommand;1bbaddfd-7a2f-489d-abda-5927dae82de6,A3ACC93B,"$$HTTPREQUEST[RestClient2]: Verb=POST, Uri=<Redacted>/v1/rest/mgmt, App=LightIngestLinux.1, User=CPC-ankur-MYBL8\ankurtyagi, ClientVersion=Runtime:{.NET_6.0.10/CLRv6.0.10/6.0.10}, ClientRequestId=KD2RunCommand;1bbaddfd-7a2f-489d-abda-5927dae82de6"
LightIngest,2023-12-21T07:02:09.7120874Z,KD.RestClient2,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94640,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,KD.RestClient.ExecuteControlCommand,KD2RunCommand;1bbaddfd-7a2f-489d-abda-5927dae82de6,21EBEC05,"$$HTTPREQUEST_RESPONSEHEADERRECEIVED[RestClient2]: Verb=POST, Uri=<Redacted>/v1/rest/mgmt, App=LightIngestLinux.1, User=CPC-ankur-MYBL8\ankurtyagi, ClientVersion=Runtime:{.NET_6.0.10/CLRv6.0.10/6.0.10}, StatusCode=OK"
LightIngest,2023-12-21T07:02:09.7123177Z,P.MonitoredActivity,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94640,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd,KD.RestClient.ExecuteControlCommand,KD2RunCommand;1bbaddfd-7a2f-489d-abda-5927dae82de6,27A39996,"MonitoredActivityCompletedSuccessfully: ActivityType=KD.RestClient.ExecuteControlCommand, Timestamp=2023-12-21T07:02:09.5027281Z, ParentActivityId=85e8b38e-71b2-4ae7-ad9f-3d0abccf54fd, Duration=209.5733 [ms], HowEnded=Success"
LightIngest,2023-12-21T07:02:09.7517462Z,P.DownloadManager,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,,,,,DA225D68,"DownloadManager3 instance 'DownloadManager3_LightIngest' is starting"
LightIngest,2023-12-21T07:02:09.7614523Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,39d2eecb-344b-434c-a9b4-3c273e233aad,39d2eecb-344b-434c-a9b4-3c273e233aad,KD.RestClient.ExecuteControlCommand,KD2RunCommand;47b1e1df-66eb-4d94-884f-b9dba1f5543a,9EAF0EFA,"AuthenticationMetadataProvider.GetAadAuthenticationMetadataForCluster: global authentication metadata is not set. Will try to obtain metadata from cluster."
LightIngest,2023-12-21T07:02:09.7616369Z,KD.AuthenticationMetadataProvider,Information,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94598,26acad83-595b-4697-b13a-9d74557cb052,39d2eecb-344b-434c-a9b4-3c273e233aad,P.NetworkCache.Fetch.AuthenticationMetadataByClusterUrl,KD2RunCommand;47b1e1df-66eb-4d94-884f-b9dba1f5543a,698837D2,"AuthenticationMetadataProvider.ExternalGetAuthenticationMetadataForClusterAsync: Called with ClusterUrl = '<Redacted>'"
LightIngest,2023-12-21T07:02:09.7662649Z,P.FirstChanceException,Critical,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94662,,,,,55A83724,"$$ALERT[CrashHandler_OnFirstChanceException_UnrecoverableException]: System.TypeInitializationException
[TypeInitializationException]: Type=Kusto.Cloud.Platform.Utils.NativeThread, InnerException=Unable to load shared library 'kernel32.dll' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libkernel32.dll: cannot open shared object file: No such file or directory
   at Kusto.Cloud.Platform.Utils.CrashHandler.OnFirstChanceException(Object sender, FirstChanceExceptionEventArgs e)
   at Kusto.Cloud.Platform.Utils.NativeThread.SetThreadDescription(String threadDescription)
   at Kusto.Cloud.Platform.Utils.AsyncInvoker.AsyncInvokerCapsuleAction`1.LongRunningThread()
   at System.Threading.Thread.StartCallback()
"

Filename: 7408-20231221070209-CPC-ankur-MYBL8-LightIngestLinux.1-94598-7053324ac70c46718c2b82903521100d.csv

LightIngest,2023-12-21T07:02:09.7776671Z,P.Diag,Critical,CPC-ankur-MYBL8,CPC-ankur-MYBL8/LightIngestLinux.1/94598,94598,94662,,,,,CRASHING,"$$ALERT[Crash_UnrecoverableException]: +++ Crash! CrashScenario=UnrecoverableException
Crashing due to an unrecoverable exception of type System.TypeInitializationException: 
+++ Call stack:
   at Kusto.Cloud.Platform.Utils.CrashTrail..ctor(CrashScenario scenario, String message, Exception ex)
   at Kusto.Cloud.Platform.Utils.CrashHandler.CrashImpl(CrashScenario scenario, String message, Exception ex, Action`1 failAction)
   at Kusto.Cloud.Platform.Utils.CrashHandler.Crash(CrashScenario scenario, String message)
   at Kusto.Cloud.Platform.Utils.CrashHandler.OnFirstChanceException(Object sender, FirstChanceExceptionEventArgs e)
   at Kusto.Cloud.Platform.Utils.NativeThread.SetThreadDescription(String threadDescription)
   at Kusto.Cloud.Platform.Utils.AsyncInvoker.AsyncInvokerCapsuleAction`1.LongRunningThread()
   at System.Threading.Thread.StartCallback()
"

OS Details:

Linux CPC-ankur-MYBL8 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
ohadbitt commented 8 months ago

Hi thanks for opening this issue, This should be fix with the new version we will release tomorrow Also should work in previous versions

ohadbitt commented 8 months ago

new version released