Rucal-Data-Solutions / datalakefoundation

Datalakehouse Foundation
GNU General Public License v3.0
3 stars 0 forks source link

Automatically create/update a metastore view for silver tables #23

Open Gateway1024 opened 6 months ago

Gateway1024 commented 6 months ago

I want to be able to query (SQL) silver tables created by foundation. to accommodate this I manually create a spark view using this function.

def createSilverTable(path: String, name: String): Unit ={ val statement = {f"DROP TABLE IF EXISTS silver.${name};" } spark.sql(f"DROP TABLE IF EXISTS silver.${name}") spark.sql(f"CREATE TABLE silver.${name} USING DELTA LOCATION '${path}'") }