StarRocks / dbt-starrocks

dbt-starrocks contains all of the code enabling dbt to work with StarRocks
16 stars 6 forks source link

[dbt-starrocks] All macros fail with type error if profiles.yml "version" is not set. #20

Closed megri closed 6 months ago

megri commented 8 months ago

Steps to reproduce the behavior (Required)

  1. Create a seed file;
  2. Make sure that no version is specified in your project's profiles.yml;
  3. run dbt seed.

Behavior (Required)

You get this error:

15:37:58    Compilation Error in seed my_seed (seeds/my_seed.csv)
  '>' not supported between instances of 'int' and 'str'

  > in macro starrocks__create_csv_table (macros/materializations/seeds.sql)
  > called by macro create_csv_table (macros/materializations/seeds/helpers.sql)
  > called by macro materialization_seed_default (macros/materializations/seeds/seed.sql)
  > called by seed my_seed (seeds/my_seed.csv)

I ran a debugger and it seems to fail here: https://github.com/StarRocks/starrocks/blob/5ca5ff0108d6a47f17d421ae37f66ebbec5775e4/contrib/dbt-connector/dbt/adapters/starrocks/impl.py#L149

… due to this line returning a string-tuple: https://github.com/StarRocks/starrocks/blob/5ca5ff0108d6a47f17d421ae37f66ebbec5775e4/contrib/dbt-connector/dbt/adapters/starrocks/connections.py#L92

… in turn happening when version is not explicitly defined, here: https://github.com/StarRocks/starrocks/blob/5ca5ff0108d6a47f17d421ae37f66ebbec5775e4/contrib/dbt-connector/dbt/adapters/starrocks/connections.py#L136-L140

StarRocks version (Required)

3.2.2-269e832

megri commented 8 months ago

Actually, since the version string allows non-int parts like 2-269e832 above, perhaps it's better to change https://github.com/StarRocks/starrocks/blob/5ca5ff0108d6a47f17d421ae37f66ebbec5775e4/contrib/dbt-connector/dbt/adapters/starrocks/impl.py#L148 to not do str->int conversion. IIRC Python allows comparison between strings based on sort order.

alberttwong commented 6 months ago

tried it to not to the str to int conversation but we need it in other areas (eg. we have to rewrite commands based on which version of SR).

I tested in this with 3.1+ with and without version. It seems to work now.

https://github.com/StarRocks/dbt-starrocks/commit/f5ada3619885dafecae79a4a3043fc0d2c27bc58