memsql / dbt-singlestore

Apache License 2.0
7 stars 8 forks source link

usage of schema vs. database #2

Closed dataders closed 2 years ago

dataders commented 2 years ago

The README.md says this:

In SingleStore, like in MySQL, database and schema denote the same concept. Therefore, you only need to specify schema in your dbt profile. database is set to NULL internally.

This exception is thrown when:

@amychen1776 and I both included the dbt standard convention of schema: <INITIALS> and got this user-unfriendly error:

22:09:35  Encountered an error:
Runtime Error
  Cannot set database in SingleStoreRelation!

I suggest either:

  1. making that error message string more helpful;
  2. forcing users to declare a database or a schema but don't allow both; or,
  3. having the database be the thing that's included in the SingleStoreIncludePolicy, not the `schema.

https://github.com/memsql/dbt-singlestore/blob/ba744c898bb7744b88c380d2d883970f543bc4e5/dbt/adapters/singlestore/relation.py#L27-L29

https://github.com/dbeatty10/dbt-mysql/blob/96655ea9f7fca7be90c9112ce8ffbb5aac1d3716/dbt/adapters/mysql/relation.py#L14-L18

pmishchenko-ua commented 2 years ago

In order to make terminology less confusing, I decided to follow the second suggestion, and force users declare a database. I wanted to do so from the beginning, but I encountered some weird errors, so I decided to follow the path of MySQL, Clickhouse and other adapters that use schema. As it turns out, schema must be specified for certain dbt internal functions to work properly (like get_catalog) - at the stage of parsing, as far I understand. There is a downside that users must always specify it, even if it's not used.

However, as schema can be used for prefixing with very easy configuration, I hope it won't be that bad for those who don't need it. Maybe dbt core can take this scenario into account, and generate a dummy schema name if it's not specified.