Closed mlonazure closed 9 months ago
I finally resolved this by switching to databricks runtime 11.2 as opposed to 12.2. Please note that the Readme/setup says that this repo has been tested on both.
I finally resolved this by switching to databricks runtime 11.2 as opposed to 12.2. Please note that the Readme/setup says that this repo has been tested on both.
Description
df = movielens.load_spark_df(spark = spark, size="100k", schema = schema,dbutils=dbutils, local_cache_path = '/dbfs/tmp')
fails with error, com.databricks.sql.io.FileReadException: Error while reading file file:/dbfs/tmp/u.data.
In which platform does it happen?
How do we replicate the issue?
df = movielens.load_spark_df(spark = spark, size="100k", schema = schema,dbutils=dbutils, local_cache_path = '/dbfs/tmp')
Expected behavior (i.e. solution)
Load the data into the dataframe.
Other Comments
I've configured the azure databricks library like the setup file.