Open rahulunlimited opened 5 years ago
I'm running into the exact same issue. I'm using Databricks 5.2 (Spark 2.4, Scala 2.11) with Python 3. I've also noticed my file system has the model.json file, and some snapshot folders. Thanks,
the problem is the version of cdm library that only support Spark 2.3, there is a PR on the library for Spark 2.4 supporting waiting to aprove
Is someone so kind and recompile this LIB so that at least Azure Databricks 5.2 can use this library? At the moment the whole Spark - CDM Integration is broken!
Thanks
check the branc 2.4 from the repository : https://github.com/Azure/spark-cdm/tree/spark24
The display(newSalesBuyingGroupsDf) method on the Databricks notebook runs forever.
The previous salesBuyingGroupsDf = (spark.read.format("com.microsoft.cdm") .option("cdmModel", inputLocation) .option("entity", "Sales BuyingGroups") .option("appId", appID) .option("appKey", appKey) .option("tenantId", tenantID) .load()) does return the metadata from model.json file. BuyingGroupID:long BuyingGroupName:string LastEditedBy:long ValidFrom:date ValidTo:date
But the display for data frame doe not work. I have also verified that the data lake folder has the files.