LightIngest unable to authenticate Kusto using ManagedIdentity

[sumanthreddy29]

Description: Using LightIngest Linux binary making a call with 1 files using managed Identity as authentication for Kusto. Unable to authenticate the kusto getting 400 bad request even though we provided valid clientID for managed identity

Reproduce: Authenticating LightIngest using managed identity LightIngest "https://ingest-xxxxx.eastus.kusto.windows.net;Fed=True" "-mi:{MANGED_IDENTITY}" -db:trips -table:test -source:"{STORAGE_URL};{STORAGE_KEY}" -pattern:"*.json" -format:json -ignoreFirst:false -cr:10.0 -dontWait:true

Error:

LightIngest failed to receive response from endpoint at 'https://ingest-xxxxxxx.eastus.kusto.windows.net'. Error: 'ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.
Status: 400 (Bad Request)

Content:
{"error":"invalid_request","error_description":"Identity not found"}

Headers:
Server: IMDS/150.870.65.1125
Date: Fri, 26 Jan 2024 16:04:15 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 68
'
Invalid service URI specified: 'https://ingest-xxxxx.eastus.kusto.windows.net'. Please make sure you are using the correct URI and that the service is accessible.

I tried to debug the code that is trying to form the ksuto connection string and the managed identity we send through parameters is assigned to EmbeddedManagedIdentity.

if (!string.IsNullOrWhiteSpace(m_args.ConnectWithManagedIdentity))
            {
                if (kcsb.FederatedSecurity || kcsb.DstsFederatedSecurity)
                {
                    kcsb.EmbeddedManagedIdentity = m_args.ConnectWithManagedIdentity;
                }
                else
                {
                    throw new UtilsArgumentException($"Command line arguments error. 'ManagedIdentity' can only be used with federated authentication.", null);
                }
            }

I looked into Kusto connection string builder code that uses embedded identity and it's in no longer in use.

    [Obsolete("Please use WithAadSystemManagedIdentity() or WithAadUserManagedIdentity(string managedIdentityClientId) instead", false)]
    public KustoConnectionStringBuilder WithAadManagedIdentity(string embeddedManagedIdentity)
    {
        return WithAadManagedIdentityImpl(embeddedManagedIdentity);
    }

[sumanthreddy29]

I tired to make a change to the authentication logic by passing appID, secret and authority values. I was able to ingest successfully. Is it possible to add one more parameter that can use appid/appsecret to authenticate with kusto.

if (!string.IsNullOrWhiteSpace(m_args.ConnectWithManagedIdentity))
            {
                if (kcsb.FederatedSecurity || kcsb.DstsFederatedSecurity)
                {
                   //kcsb.EmbeddedManagedIdentity = m_args.ConnectWithManagedIdentity;
                    kcsb.ApplicationClientId = m_args.ConnectWithManagedIdentity;
                    kcsb.ApplicationKey = "SECRET";
                    kcsb.Authority = "Authority";
                }
                else
                {
                    throw new UtilsArgumentException($"Command line arguments error. 'ManagedIdentity' can only be used with federated authentication.", null);
                }
            }

[yogilad]

Hi @sumanthreddy29, You can use App / Key or App Certificate authentication using the connection string. See: https://learn.microsoft.com/en-us/azure/data-explorer/kusto/api/connection-strings/kusto

As per the first issue, the errors suggests the managed identity you referred to is not assigned to your linux machine. 'ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource.

Can you please double check that,

1. A managed identity is assigned to your machine
2. You have provided the correct managed identity client id
3. If the managed identity is a System Assigned MI you have used the keyword "system" instead of it's client id (it technically does not have one)

[sumanthreddy29]

Hi @yogilad ,

Thanks for the suggestion. I tried using the App/ Key authentication using Kusto connection string and it worked. Thanks