risingwavelabs / dbt-risingwave

Apache License 2.0
21 stars 6 forks source link

risingwave's driver always maps varchar to varchar(0) #8

Closed neverchanje closed 1 year ago

neverchanje commented 1 year ago

{% macro risingwave__get_columns_in_relation(relation) -%}
  {% call statement('get_columns_in_relation', fetch_result=True) %}
      select
          column_name,
          data_type,
          0 as character_maximum_length, -- todo
          0 as numeric_precision,
          0 as numeric_scale

      from {{ relation.information_schema('columns') }}
      where table_name = '{{ relation.identifier }}'
        {% if relation.schema %}
        and table_schema = '{{ relation.schema }}'
        {% endif %}
      order by ordinal_position

  {% endcall %}
  {% set table = load_result('get_columns_in_relation').table %}
  {{ return(sql_convert_columns_in_relation(table)) }}
{% endmacro %}

It's because that in the postgres driver where risingwave's driver derives, character_maximum_length is a required param.

chenzl25 commented 1 year ago

Could you please give an example to reproduce it?

neverchanje commented 1 year ago
{%- set cols = adapter.get_columns_in_relation(source('public', 'shopify_orders')) -%}
WITH source AS (
    SELECT
        *
    FROM
        {{ source(
            'public',
            'shopify_orders'
        ) }}
)
SELECT
    {% for col in cols %}
        {% if col.name %}
            CAST(
                {{ col.name }} AS {{ col.data_type }}
            ) AS {{ col.name }}

            {% if not loop.last %},
            {% endif %}
        {% endif %}
    {% endfor %}
FROM
    source

Here is the example. First you'll need to define the source's schema in a yaml file. And even if the column type I used is varchar, get_columns_in_relation still returns varchar(0).

SELECT

            CAST(
                id AS character varying(0)
            ) AS id

            ,

The above is the compiled code. I've searched but found no alternative for get_columns_in_relation.

fuyufjh commented 1 year ago

What does postgres-dbt returns for varchar without length?

image