search

Azure-Samples / cdm-azure-data-services-integration

Tutorials and sample code for integrating CDM folders with Azure Data Services
MIT License
70 stars 46 forks source link

databricks sample problem #13

Open etshao1947 opened 5 years ago

etshao1947 commented 5 years ago

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

in Azure databricks, after reading a CMD into a df, 'display(df)' never finishes.

DimitriCabaud commented 5 years ago

Same issue

adiazcan commented 5 years ago

This is a problem with the cdm library that only work with Spark 2.3 and the latest Databricks only have Spark 2.4.

bvsrini commented 5 years ago

I tried both CDM library "spark-cdm-assembly-0.3.jar" in "https://github.com/Azure-Samples/cdm-azure-data-services-integration/tree/master/AzureDatabricks/Library" and the one on Azure samples. I am getting the conversion errors to utf-8 or to int. This is failing both for display() and for save().