Maven coordinates not found in Databricks Install libraries - com.microsoft.azure.kusto:kusto-spark_3.0_2.12:4.0.2

[ravikiransharvirala]

Describe the bug

Maven coordinates not found in Databricks Install libraries - com.microsoft.azure.kusto:kusto-spark_3.0_2.12:4.0.2

[ag-ramachandran]

@ravikiransharvirala

2 points:

a) The release was made only yesterday and will take time to reflect on Maven Central.

b) You are using Databricks and has to be set up specifically to use 4.0.x versions that are currently using JDK 11. The same fixes are available on 3.x series and is available on Sonatype nexus (maven staging)

https://repo1.maven.org/maven2/com/microsoft/azure/kusto/kusto-spark_3.0_2.12/3.1.16/

Sonatype : https://central.sonatype.com/artifact/com.microsoft.azure.kusto/kusto-spark_3.0_2.12/4.0.1/versions

If either does not work (network restrictions if any), the uber jar is available on releases, for test runs a good start would be to upload the jars to databricks and run the tests.

https://github.com/Azure/azure-kusto-spark/releases/tag/v3.0_3.1.16

Feel free to reach-out or re-open in case you have questions.

[ravikiransharvirala]

Thank you so much @ag-ramachandran.

I have one more question regarding the connector.

• I'm able to use the connector to ingest data and query it when my Kusto uri ends with .kusto.windows.net. But I get a below error when I use a cluster with uri that ends z2.kusto.fabric.microsoft.com

Py4JJavaError: An error occurred while calling o1338.load. : com.microsoft.azure.kusto.data.exceptions.DataClientException: $$ALERT[ValidateHostnameIsTrusted]: Can't communicate with '<cluster-name>.z2.kusto.fabric.microsoft.com' as this hostname is currently not trusted; please see https://aka.ms/kustotrustedendpoints

Could you please help me with issue?

(Apologies for re-opening this case even though it's not related to original question)

[ag-ramachandran]

Quick question @ravikiransharvirala, which version are you using currently

[ravikiransharvirala]

@ag-ramachandran I'm using com.microsoft.azure.kusto:kusto-spark_3.0_2.12:3.1.15

[ag-ramachandran]

Ok, this needs work to be backported to the 3.x series. The Fabric URL was whitelisted on this PR. This means you will need to use 4.0.1

This means 2 things

a) Add the jar file 4.0.2 from https://github.com/Azure/azure-kusto-spark/releases/tag/v3.0_4.0.2

b) set JNAME=zulu11-ca-amd64 as this needs JDK 11 to run - shown below (we will make it compatible for JDK 8 soon with a new release that is in flight)

[ravikiransharvirala]

@ag-ramachandran Thank you so much. The solution you recommended worked.