Closed RBirdwatcher closed 1 month ago
One advantage of the default use of SQLite is that it is stored in one file and we can emial it or send it to someone else, or easily store it in Github. So perhaps SQLite is the better default to use for developement.
Certainly in production use Postgres will be more popular, so we can consider ensuring both are possible. When we create test data it is good to store as editable SQL statemetns also, we should ensure these are fit for use on Postgres which might have slightly different syntax (e.g. use of quotes) in insert statements
Out of Scope
As we aim for Spark usage also, this link may be useful:
https://www.machinelearningplus.com/pyspark/pyspark-connect-to-postgresql/?utm_content=cmp-true