Package fails to inference type of column for PostgreSQL data_types

vbatychko-modeln commented 11 months ago

DBT's data_type can have more broader set of types that are currently I see in package's source. The type can be any of supported in underlying database. For exple, PostgreSQL varchar type:

Error: invalid operation: Error while calling method: Python error: RuntimeError: Unknown column type of volumes.id: varchar(255)

Package version: 0.6.0

jronsse commented 11 months ago

Hello !

We kinda have the same issue:

Context

We are currently experimenting with Cube Cloud after a month or so testing things out with Cube Core. We plan on using dbt to generate our models. We recently updated our process and now include columns' data_type in the generated YAML model files. We use dbt-codegen: generate_model_yaml to generate the model. We use Databricks SQL Serverless as our data warehouse

Issue encountered

The resulting manifest.json mentions data types that are not supported by cube-dbt:

"store_indice": {
  "name": "store_indice",
  "description": "",
  "meta": {},
  "data_type": "double",
  "constraints": [],
  "quote": null,
  "tags": []
},

Here's a list of datatypes that we might encounter in the manifest

@igorlukanin Would you agree that adding those type aliases to cube-dbt is the way to go? If so I'd be happy to create a PR.

Thanks !

bzenker-amplify commented 4 months ago

I'm not sure that this qualifies as an issue of the cube_dbt codebase. Cube clarifies what data types are allowed, and the list is abysmally small. So, all that cube_dbt is doing is supporting that list.

You should open an issue on https://github.com/cube-js/cube if you want them to support more data types.

jmkacz commented 3 weeks ago

Looking at this, I wonder about the mapping dict in column,py. It is currently hardcoded. Could this be exposed as a configuration file?

cube-js / cube_dbt

Package fails to inference type of column for PostgreSQL data_types #5

Context

Issue encountered