search

databricks / Spark-The-Definitive-Guide

Spark: The Definitive Guide's Code Repository
http://shop.oreilly.com/product/0636920034957.do
Other
2.85k stars 2.76k forks source link

Ch9 - Writing to SQL database - SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info) #67

Open shanmugavel04 opened 3 years ago

shanmugavel04 commented 3 years ago

When executing the below command to write to the below path facing the issue

val newPath = "jdbc:sqlite://tmp/my-sqlite.db" val tablename = "flight_info" val props = new java.util.Properties props.setProperty("driver", "org.sqlite.JDBC") csvFile.write.mode("overwrite").jdbc(newPath, tablename, props)

Driver Stack trace at high level: Caused by: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info) at org.sqlite.core.DB.newSQLException(DB.java:890) at org.sqlite.core.DB.newSQLException(DB.java:901) at org.sqlite.core.DB.throwex(DB.java:868) at org.sqlite.core.NativeDB.prepare(Native Method) at org.sqlite.core.DB.prepare(DB.java:211)

could you please review and let us know how to resolve this issue further?

shanmugavel04 commented 3 years ago

I think SQLite will not necessarily work well in a distributed setting because of its requirement to lock the entire database on write. could you please confirm my understanding?