search

Azure-Samples / cdm-azure-data-services-integration

Tutorials and sample code for integrating CDM folders with Azure Data Services
MIT License
70 stars 46 forks source link

The display(newSalesBuyingGroupsDf) method on databricks notebook runs forever #9

Open rahulunlimited opened 5 years ago

rahulunlimited commented 5 years ago

The display(newSalesBuyingGroupsDf) method on the Databricks notebook runs forever.

The previous salesBuyingGroupsDf = (spark.read.format("com.microsoft.cdm") .option("cdmModel", inputLocation) .option("entity", "Sales BuyingGroups") .option("appId", appID) .option("appKey", appKey) .option("tenantId", tenantID) .load()) does return the metadata from model.json file. BuyingGroupID:long BuyingGroupName:string LastEditedBy:long ValidFrom:date ValidTo:date

But the display for data frame doe not work. I have also verified that the data lake folder has the files.

tonio-lora commented 5 years ago

I'm running into the exact same issue. I'm using Databricks 5.2 (Spark 2.4, Scala 2.11) with Python 3. I've also noticed my file system has the model.json file, and some snapshot folders. Thanks,

adiazcan commented 5 years ago

the problem is the version of cdm library that only support Spark 2.3, there is a PR on the library for Spark 2.4 supporting waiting to aprove

Guido-Jacobs-MS commented 5 years ago

Is someone so kind and recompile this LIB so that at least Azure Databricks 5.2 can use this library? At the moment the whole Spark - CDM Integration is broken!

Thanks

Mimetis commented 5 years ago

check the branc 2.4 from the repository : https://github.com/Azure/spark-cdm/tree/spark24