search

databricks / spark-sql-perf

Apache License 2.0
582 stars 406 forks source link

AnalysisException when calling genData #173

Open parsifal-47 opened 5 years ago

parsifal-47 commented 5 years ago

Hi, on Spark 2.3.2:

scala> tables.genData(
     |     location = rootDir,
     |     format = format,
     |     overwrite = true, // overwrite the data that is already there
     |     partitionTables = true, // create the partitioned fact tables
     |     clusterByPartitionColumns = true, // shuffle to get partitions coalesced into single files.
     |     filterOutNullPartitionValues = false, // true to filter out the partition with NULL key value
     |     tableFilter = "", // "" means generate all tables
     |     numPartitions = 1) 
org.apache.spark.sql.AnalysisException: cannot resolve '`cs_sold_date_sk`' given input columns: [catalog_sales_text.value]; line 8 pos 2;
'RepartitionByExpression ['cs_sold_date_sk], 200
+- Project [value#64]
   +- SubqueryAlias catalog_sales_text
      +- LogicalRDD [value#64], false