Unable to load Kusto.Data.dll in Windows PowerShell 5.1: unable to find "Microsoft.WindowsAzure.Security.Authentication"

[jazzdelightsme]

The version 12.0.0 nuget package has a couple of versions under the tools directory: net6.0 and net472. I am trying to load Kusto.Data.dll from the net472 directory in Windows PowerShell 5.1 (the built-in powershell.exe), but it fails with the following error:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

I can't figure out what package would supply that dependency.

(Loading the net6.0 version in pwsh.exe v7.4 works fine; but the net6.0 version can't be loaded in powershell.exe 5.1, because it is running on an older framework.)

[ohadbitt]

This is the old dsts dll missing. As you are using the newest client i would advise you using modern dsts authentication. please refer to these docs on how to setup dsts v2 https://kusto.azurewebsites.net/docs/kusto/management/access-control/dsts.html#modern-dsts-jwt-tokens-authentication If you read this and still want to use classic dSTS you will also find in the docs above how to connect to the internal feed and use old dsts.

[jazzdelightsme]

Thanks for the info, @ohadbitt! I definitely don't want to use dSTS; I didn't even know what it was. What I am trying to accomplish is use Kusto.Data.dll in legacy Windows PowerShell 5.1 (powershell.exe). To do this, I must use the copy under tools\net472. When I use that version, and try to execute a query, it apparently attempts to use dSTS of its own volition. (When I run the same code in modern pwsh.exe, using the tools\net6.0 version of Kusto.Data.dll, everything works fine.)

Is this a bug in the net472 version, that it is trying to use dSTS instead of something better?

[yogilad]

@jazzdelightsme ,

1. please try using 12.0.1 since we fixed some dll dependecy issues
2. Can you share the connection string you use and how you initialize KustoConnectionStringBuilder?

[jazzdelightsme]

Thanks, @yogilad.

1. No difference with 12.0.1.
2. I don't think you are understanding the problem: I cannot initialize KustoConnectionStringBuilder, because I cannot even load the Kusto.Data.dll binary!

The problem here is that the net472 version of Kusto.Data.dll cannot be used by anyone outside of Microsoft (because they will not have access to the old "classic dSTS"). As MSFT, I could get access to dSTS, but I don't want to. ;)

[yogilad]

@jazzdelightsme , the version fixed an issue with loading the library in powershell, so we wanted to make sure this issue is not in the way.

DLLs in C# are only loaded when needed, so if you don't have dSTS dlls and you don't hit any code paths that use them you should be good. Per our design and tests we conducted, no dSTS code is called unless the user tries to authenticate with dSTS.

Can you share your code concerning connection string building and the connection string you use? (please hide any confidential values, but keep the used params clear)

[jazzdelightsme]

Thanks @yogilad, I think I confused a different problem with this one.

Here is the boiled-down code that resulted in a failure because it could not load dSTS:

$clusterUrl = 'https://thething.kusto.windows.net;Fed=True'
$databaseName = 'TheDbName'
$kcsb = [Kusto.Data.KustoConnectionStringBuilder]::new($clusterUrl, $databaseName)

$queryProvider = [Kusto.Data.Net.Client.KustoClientFactory]::CreateCslQueryProvider($kcsb)

$crp = [Kusto.Data.Common.ClientRequestProperties]::new()
$crp.ClientRequestId = "MyKustoQuery." + [Guid]::NewGuid().ToString()
Write-Verbose "KQL client request ID: $($crp.ClientRequestId)"
$crp.SetOption([Kusto.Data.Common.ClientRequestProperties]::OptionServerTimeout, [TimeSpan]::FromSeconds(30))

$query = 'SomeEvents
 | where _name has "TheNameIAmInterestedIn"
 | project _data
'
#   Execute the query
$reader = $queryProvider.ExecuteQuery($query, $crp)

$dataTable = [Kusto.Cloud.Platform.Data.ExtendedDataReader]::ToDataSet($reader).Tables[0]
$dataView = [System.Data.DataView]::new($dataTable)

Note that the failure only happens when running on .NET Framework (in Windows PowerShell 5.1)--if we run in PowerShell 7 (on .NET 6 or 8), using the net6.0 Kusto binaries, then this same code works fine.

[yogilad]

Does this still fail in the when loading Kusto.Data or does it fail in one of the lines in the above code example? If so, which one? Do you call stack or STDOUT to show?

[jazzdelightsme]

It fails when loading Kusto.Data.dll.

The repro is to run powershell.exe (v5.1, built in to Windows), then run Add-Type -LiteralPath "path\to\net472\Kusto.Data.dll". The result is:

Add-Type: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

Dumping the LoaderExceptions property:

Message        : Could not load file or assembly 'Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
FileName       : Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
FusionLog      : WRN: Assembly binding logging is turned OFF.
                 To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
                 Note: There is some performance penalty associated with assembly bind failure logging.
                 To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

Data           : {}
InnerException :
TargetSite     :
StackTrace     :
HelpLink       :
Source         :
HResult        : -2147024894

(No stack trace, unfortunately.)

I tried again, but this time with the Fusion logging turned on, in case that is helpful (I don't think it is, though--it is just telling us what we know: the dSTS binary can't be found).

PS T:\os4\src> $error[0].Exception.LoaderExceptions | Format-List * -Force

Message        : Could not load file or assembly 'Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
FileName       : Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
FusionLog      : === Pre-bind state information ===
                 LOG: DisplayName = Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
                  (Fully-specified)
                 LOG: Appbase = file:///C:/WINDOWS/System32/WindowsPowerShell/v1.0/
                 LOG: Initial PrivatePath = NULL
                 Calling assembly : Kusto.Data, Version=12.0.1.0, Culture=neutral, PublicKeyToken=829a00acde12d551.
                 ===
                 LOG: This bind starts in LoadFrom load context.
                 WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with
                 Assembly.Load().
                 LOG: Using application configuration file: C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe.Config
                 LOG: Using host configuration file:
                 LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
                 LOG: Post-policy reference: Microsoft.WindowsAzure.Security.Authentication, Version=3.0.0.0, Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35
                 LOG: Attempting download of new URL file:///C:/WINDOWS/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Security.Authentication.DLL.
                 LOG: Attempting download of new URL file:///C:/WINDOWS/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Security.Authentication/Micros
                 oft.WindowsAzure.Security.Authentication.DLL.
                 LOG: Attempting download of new URL file:///C:/WINDOWS/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Security.Authentication.EXE.
                 LOG: Attempting download of new URL file:///C:/WINDOWS/System32/WindowsPowerShell/v1.0/Microsoft.WindowsAzure.Security.Authentication/Micros
                 oft.WindowsAzure.Security.Authentication.EXE.
                 LOG: Attempting download of new URL file:///C:/Users/danthom/Documents/PSModules/ixptools/WindowsDevTools/packages/microsoft.azure.kusto.too
                 ls/12.0.1/tools/net472/Microsoft.WindowsAzure.Security.Authentication.DLL.
                 LOG: Attempting download of new URL file:///C:/Users/danthom/Documents/PSModules/ixptools/WindowsDevTools/packages/microsoft.azure.kusto.too
                 ls/12.0.1/tools/net472/Microsoft.WindowsAzure.Security.Authentication/Microsoft.WindowsAzure.Security.Authentication.DLL.
                 LOG: Attempting download of new URL file:///C:/Users/danthom/Documents/PSModules/ixptools/WindowsDevTools/packages/microsoft.azure.kusto.too
                 ls/12.0.1/tools/net472/Microsoft.WindowsAzure.Security.Authentication.EXE.
                 LOG: Attempting download of new URL file:///C:/Users/danthom/Documents/PSModules/ixptools/WindowsDevTools/packages/microsoft.azure.kusto.too
                 ls/12.0.1/tools/net472/Microsoft.WindowsAzure.Security.Authentication/Microsoft.WindowsAzure.Security.Authentication.EXE.

Data           : {}
InnerException :
TargetSite     :
StackTrace     :
HelpLink       :
Source         :
HResult        : -2147024894

[AsafMah]

Instead of using Add-Type, try loading via:

[System.Reflection.Assembly]::LoadFrom(".\Kusto.Data.dll")

[jazzdelightsme]

If I use LoadFrom instead of Add-Type, the load itself appears to succeed. But then later, when I try to execute a query, I get this error instead:

PS T:\os3\src> $reader = $queryProvider.ExecuteQuery($q, $crp)
MethodInvocationException: Exception calling "ExecuteQuery" with "2" argument(s): "Method not found: '!0
Microsoft.Identity.Client.BaseAbstractApplicationBuilder`1.WithLogging(Microsoft.IdentityModel.Abstractions.IIdentityLogger, Boolean)'."

Here is more information about that exception:

Message        : Method not found: '!0 Microsoft.Identity.Client.BaseAbstractApplicationBuilder`1.WithLogging(Microsoft.IdentityModel.Abstractions.IIdentityLogger, Boolean)'.
Data           : {}
InnerException :
TargetSite     : Void MoveNext()
StackTrace     :    at Kusto.Cloud.Platform.Http.AadUserHttpClientAuthenticator.<GetTokenAsync>d__8.MoveNext()
                    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
                    at Kusto.Cloud.Platform.Http.AadUserHttpClientAuthenticator.GetTokenAsync()
                    at Kusto.Cloud.Platform.Http.AadUserHttpClientAuthenticator.<AuthenticateAsync>d__7.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at Kusto.Data.Net.Client.RestClient2.<MakeHttpRequestAsyncImpl>d__60.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at Kusto.Cloud.Platform.Utils.MonitoredActivity.<InvokeAsync>d__12`2.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at Kusto.Cloud.Platform.Utils.MonitoredActivity.<InvokeAsync>d__12`2.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at Kusto.Data.Net.Client.RestClient2.<MakeHttpRequestAsync>d__58.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at Kusto.Data.Net.Client.RestClient2.<ExecuteQueryAsync>d__45.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at CallSite.Target(Closure , CallSite , Object , Object , Object )

[AsafMah]

Can you try version 12.1.2?

[AsafMah]

Version 12.2.3 has further powershell fixes, can you provide an update?

[alexanderbob]

I am facing the same error with powershell 7.4.5 and net6 build of Azure.Kusto.Tools 12.2.7 Tried loading with both [System.Reflection.Assembly]::LoadFrom() and Add-Type

Exception calling "ExecuteQuery" with "2" argument(s): "Method not found: '!0 Microsoft.Identity.Client.BaseAbstractApplicationBuilder`1.WithLogging(Microsoft.IdentityModel.Abstractions.IIdentityLogger, Boolean)'.