Open ashikaumanga opened 4 years ago
@ashikaumanga I got the same issue with delta, I solve it by overriding conf
method.
class MyDeltaTest extends FunSuite with DataFrameSuiteBase {
...
override def conf: SparkConf = super.conf
.set("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
.set("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
...
}
I am trying to unit-test DeltaLake operations. For this I need to initialize SparkSession with following options
However,when I run the unit-tests it still throws following error, probably caused by not setting above options to the SparkSession
Anyway to fix this ?