search

StarRocks / dbt-starrocks

dbt-starrocks contains all of the code enabling dbt to work with StarRocks
20 stars 8 forks source link

Pass DBT basic unit test adapter suite #32

Open alberttwong opened 8 months ago

alberttwong commented 8 months ago 
CLIENT: Server listening on port 65417...
Received JSON data in run script
Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/Users/atwong/sandbox/dbt-starrocks', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestSingularTestsMyAdapter::test_singular_tests', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestSingularTestsEphemeralMyAdapter::test_singular_tests_ephemeral', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestEmptyMyAdapter::test_empty', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestEphemeralMyAdapter::test_ephemeral', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestIncrementalMyAdapter::test_incremental', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestGenericTestsMyAdapter::test_generic_tests', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestSnapshotCheckColsMyAdapter::test_snapshot_check_cols', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestSnapshotTimestampMyAdapter::test_snapshot_timestamp', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestBaseAdapterMethod::test_adapter_methods']
============================= test session starts ==============================
platform darwin -- Python 3.9.16, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/atwong/sandbox/dbt-starrocks
configfile: pytest.ini
plugins: dotenv-0.5.2
collected 9 items

tests/functional/adapter/test_basic.py ......FF.                         [100%]

=================================== FAILURES ===================================
___________ TestSnapshotCheckColsMyAdapter.test_snapshot_check_cols ____________

self = <test_basic.TestSnapshotCheckColsMyAdapter object at 0x10ce45310>
project = <dbt.tests.fixtures.project.TestProjInfo object at 0x10d370c40>

    def test_snapshot_check_cols(self, project):
        # seed command
        results = run_dbt(["seed"])
        assert len(results) == 2

        # snapshot command
>       results = run_dbt(["snapshot"])

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/adapter/basic/test_snapshot_check_cols.py:44: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['snapshot', '--project-dir', '/private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/project6', '--profiles-dir', '/private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/profile6']
expect_pass = True

    def run_dbt(
        args: Optional[List[str]] = None,
        expect_pass: bool = True,
    ):
        # Ignore logbook warnings
        warnings.filterwarnings("ignore", category=DeprecationWarning, module="logbook")

        # reset global vars
        reset_metadata_vars()

        # The logger will complain about already being initialized if
        # we don't do this.
        log_manager.reset_handlers()
        if args is None:
            args = ["run"]

        print("\n\nInvoking dbt with {}".format(args))
        from dbt.flags import get_flags

        flags = get_flags()
        project_dir = getattr(flags, "PROJECT_DIR", None)
        profiles_dir = getattr(flags, "PROFILES_DIR", None)
        if project_dir and "--project-dir" not in args:
            args.extend(["--project-dir", project_dir])
        if profiles_dir and "--profiles-dir" not in args:
            args.extend(["--profiles-dir", profiles_dir])

        dbt = dbtRunner()
        res = dbt.invoke(args)

        # the exception is immediately raised to be caught in tests
        # using a pattern like `with pytest.raises(SomeException):`
        if res.exception is not None:
            raise res.exception

        if expect_pass is not None:
>           assert res.success == expect_pass, "dbt exit state did not match expected"
E           AssertionError: dbt exit state did not match expected

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/util.py:108: AssertionError
---------------------------- Captured stdout setup -----------------------------

=== Test project_root: /private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/project6
----------------------------- Captured stdout call -----------------------------

Invoking dbt with ['seed']
23:40:48  Running with dbt=1.6.2
23:40:48  Registered adapter: starrocks=1.4.2
23:40:48  Unable to do partial parsing because saved manifest not found. Starting full parse.
23:40:48  Found 3 snapshots, 2 seeds, 0 sources, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
23:40:48  
23:40:49  Concurrency: 1 threads (target='default')
23:40:49  
23:40:49  1 of 2 START seed file test17102868486143503769_test_basic.added ............... [RUN]
23:40:49  1 of 2 OK loaded seed file test17102868486143503769_test_basic.added ........... [INSERT 20 in 0.22s]
23:40:49  2 of 2 START seed file test17102868486143503769_test_basic.base ................ [RUN]
23:40:49  2 of 2 OK loaded seed file test17102868486143503769_test_basic.base ............ [INSERT 10 in 0.19s]
23:40:49  
23:40:49  Finished running 2 seeds in 0 hours 0 minutes and 0.47 seconds (0.47s).
23:40:49  
23:40:49  Completed successfully
23:40:49  
23:40:49  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2

Invoking dbt with ['snapshot']
23:40:49  Running with dbt=1.6.2
23:40:49  Registered adapter: starrocks=1.4.2
23:40:49  Found 3 snapshots, 2 seeds, 0 sources, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
23:40:49  
23:40:49  Concurrency: 1 threads (target='default')
23:40:49  
23:40:49  1 of 3 START snapshot test17102868486143503769_test_basic.cc_all_snapshot ...... [RUN]
23:40:49  1 of 3 ERROR snapshotting test17102868486143503769_test_basic.cc_all_snapshot .. [ERROR in 0.04s]
23:40:49  2 of 3 START snapshot test17102868486143503769_test_basic.cc_date_snapshot ..... [RUN]
23:40:49  2 of 3 ERROR snapshotting test17102868486143503769_test_basic.cc_date_snapshot . [ERROR in 0.01s]
23:40:49  3 of 3 START snapshot test17102868486143503769_test_basic.cc_name_snapshot ..... [RUN]
23:40:49  3 of 3 ERROR snapshotting test17102868486143503769_test_basic.cc_name_snapshot . [ERROR in 0.04s]
23:40:49  
23:40:49  Finished running 3 snapshots in 0 hours 0 minutes and 0.14 seconds (0.14s).
23:40:49  
23:40:49  Completed with 3 errors and 0 warnings:
23:40:49  
23:40:49    Compilation Error in snapshot cc_all_snapshot (snapshots/cc_all_snapshot.sql)
  macro 'dbt_macro__snapshot_check_all_get_existing_columns' takes not more than 2 argument(s)

  > in macro snapshot_check_strategy (macros/materializations/snapshots/strategies.sql)
  > called by macro materialization_snapshot_starrocks (macros/materializations/snapshot/snapshot.sql)
  > called by snapshot cc_all_snapshot (snapshots/cc_all_snapshot.sql)
23:40:49  
23:40:49    Compilation Error in snapshot cc_date_snapshot (snapshots/cc_date_snapshot.sql)
  macro 'dbt_macro__snapshot_check_all_get_existing_columns' takes not more than 2 argument(s)

  > in macro snapshot_check_strategy (macros/materializations/snapshots/strategies.sql)
  > called by macro materialization_snapshot_starrocks (macros/materializations/snapshot/snapshot.sql)
  > called by snapshot cc_date_snapshot (snapshots/cc_date_snapshot.sql)
23:40:49  
23:40:49    Compilation Error in snapshot cc_name_snapshot (snapshots/cc_name_snapshot.sql)
  macro 'dbt_macro__snapshot_check_all_get_existing_columns' takes not more than 2 argument(s)

  > in macro snapshot_check_strategy (macros/materializations/snapshots/strategies.sql)
  > called by macro materialization_snapshot_starrocks (macros/materializations/snapshot/snapshot.sql)
  > called by snapshot cc_name_snapshot (snapshots/cc_name_snapshot.sql)
23:40:49  
23:40:49  Done. PASS=0 WARN=0 ERROR=3 SKIP=0 TOTAL=3
____________ TestSnapshotTimestampMyAdapter.test_snapshot_timestamp ____________

self = <test_basic.TestSnapshotTimestampMyAdapter object at 0x10ce45bb0>
project = <dbt.tests.fixtures.project.TestProjInfo object at 0x10d3c5970>

    def test_snapshot_timestamp(self, project):
        # seed command
        results = run_dbt(["seed"])
        assert len(results) == 3

        # snapshot command
        results = run_dbt(["snapshot"])
        assert len(results) == 1

        # snapshot has 10 rows
        check_relation_rows(project, "ts_snapshot", 10)

        # point at the "added" seed so the snapshot sees 10 new rows
>       results = run_dbt(["-d", "snapshot", "--vars", "seed_name: added"])

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/adapter/basic/test_snapshot_timestamp.py:49: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['snapshot', '--vars', 'seed_name: added', '--project-dir', '/private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/project7', '--profiles-dir', ...]
expect_pass = True

    def run_dbt(
        args: Optional[List[str]] = None,
        expect_pass: bool = True,
    ):
        # Ignore logbook warnings
        warnings.filterwarnings("ignore", category=DeprecationWarning, module="logbook")

        # reset global vars
        reset_metadata_vars()

        # The logger will complain about already being initialized if
        # we don't do this.
        log_manager.reset_handlers()
        if args is None:
            args = ["run"]

        print("\n\nInvoking dbt with {}".format(args))
        from dbt.flags import get_flags

        flags = get_flags()
        project_dir = getattr(flags, "PROJECT_DIR", None)
        profiles_dir = getattr(flags, "PROFILES_DIR", None)
        if project_dir and "--project-dir" not in args:
            args.extend(["--project-dir", project_dir])
        if profiles_dir and "--profiles-dir" not in args:
            args.extend(["--profiles-dir", profiles_dir])

        dbt = dbtRunner()
        res = dbt.invoke(args)

        # the exception is immediately raised to be caught in tests
        # using a pattern like `with pytest.raises(SomeException):`
        if res.exception is not None:
            raise res.exception

        if expect_pass is not None:
>           assert res.success == expect_pass, "dbt exit state did not match expected"
E           AssertionError: dbt exit state did not match expected

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/util.py:108: AssertionError
---------------------------- Captured stdout setup -----------------------------

=== Test project_root: /private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/project7
----------------------------- Captured stdout call -----------------------------

Invoking dbt with ['seed']
23:40:49  Running with dbt=1.6.2
23:40:49  Registered adapter: starrocks=1.4.2
23:40:49  Unable to do partial parsing because saved manifest not found. Starting full parse.
23:40:50  Found 1 snapshot, 3 seeds, 0 sources, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
23:40:50  
23:40:50  Concurrency: 1 threads (target='default')
23:40:50  
23:40:50  1 of 3 START seed file test17102868504212522810_test_basic.added ............... [RUN]
23:40:50  1 of 3 OK loaded seed file test17102868504212522810_test_basic.added ........... [INSERT 20 in 0.24s]
23:40:50  2 of 3 START seed file test17102868504212522810_test_basic.base ................ [RUN]
23:40:50  2 of 3 OK loaded seed file test17102868504212522810_test_basic.base ............ [INSERT 10 in 0.19s]
23:40:50  3 of 3 START seed file test17102868504212522810_test_basic.newcolumns .......... [RUN]
23:40:51  3 of 3 OK loaded seed file test17102868504212522810_test_basic.newcolumns ...... [INSERT 10 in 0.21s]
23:40:51  
23:40:51  Finished running 3 seeds in 0 hours 0 minutes and 0.69 seconds (0.69s).
23:40:51  
23:40:51  Completed successfully
23:40:51  
23:40:51  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Invoking dbt with ['snapshot']
23:40:51  Running with dbt=1.6.2
23:40:51  Registered adapter: starrocks=1.4.2
23:40:51  Found 1 snapshot, 3 seeds, 0 sources, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
23:40:51  
23:40:51  Concurrency: 1 threads (target='default')
23:40:51  
23:40:51  1 of 1 START snapshot test17102868504212522810_test_basic.ts_snapshot .......... [RUN]
23:40:51  1 of 1 OK snapshotted test17102868504212522810_test_basic.ts_snapshot .......... [SUCCESS 10 in 0.20s]
23:40:51  
23:40:51  Finished running 1 snapshot in 0 hours 0 minutes and 0.24 seconds (0.24s).
23:40:51  
23:40:51  Completed successfully
23:40:51  
23:40:51  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Invoking dbt with ['-d', 'snapshot', '--vars', 'seed_name: added']
23:40:51  Running with dbt=1.6.2
23:40:51  running dbt with arguments {'printer_width': '80', 'indirect_selection': 'eager', 'log_cache_events': 'False', 'write_json': 'True', 'partial_parse': 'True', 'cache_selected_only': 'False', 'profiles_dir': '/private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-97/profile7', 'version_check': 'True', 'warn_error': 'None', 'log_path': '/Users/atwong/sandbox/dbt-starrocks/logs/test17102868504212522810', 'debug': 'True', 'fail_fast': 'False', 'use_colors': 'True', 'use_experimental_parser': 'False', 'no_print': 'None', 'quiet': 'False', 'log_format': 'default', 'invocation_command': 'dbt --rootdir=/Users/atwong/sandbox/dbt-starrocks', 'warn_error_options': 'WarnErrorOptions(include=[], exclude=[])', 'static_parser': 'True', 'target_path': 'None', 'introspect': 'True', 'send_anonymous_usage_stats': 'False'}
23:40:51  Connection '_test' was properly closed.
23:40:51  Registered adapter: starrocks=1.4.2
23:40:51  checksum: 28af41976244efb18c98fa3cfba69f85a75f3346c2f9b2bb706d0954f8ab0ad6, vars: {'seed_name': 'added'}, profile: , target: , version: 1.6.2
23:40:51  Unable to do partial parsing because config vars, config profile, or config target have changed
23:40:51  previous checksum: 28af41976244efb18c98fa3cfba69f85a75f3346c2f9b2bb706d0954f8ab0ad6, current checksum: 8d66c46a7c5c0e5717f369273dbde9db64740684d9ba42e6442c1808d698ae9f
23:40:51  Found 1 snapshot, 3 seeds, 0 sources, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
23:40:51  
23:40:51  Acquiring new starrocks connection 'master'
23:40:51  Acquiring new starrocks connection 'list_schemas'
23:40:51  Using starrocks connection "list_schemas"
23:40:51  On list_schemas: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "connection_name": "list_schemas"} */
select distinct schema_name from information_schema.schemata
23:40:51  Opening a new connection, currently in state init
23:40:51  SQL status: SUCCESS 8 in 0.0 seconds
23:40:51  On list_schemas: Close
23:40:51  Re-using an available connection from the pool (formerly list_schemas, now list_None_test17102868504212522810_test_basic)
23:40:51  Using starrocks connection "list_None_test17102868504212522810_test_basic"
23:40:51  On list_None_test17102868504212522810_test_basic: BEGIN
23:40:51  Opening a new connection, currently in state closed
23:40:51  SQL status: SUCCESS 0 in 0.0 seconds
23:40:51  Using starrocks connection "list_None_test17102868504212522810_test_basic"
23:40:51  On list_None_test17102868504212522810_test_basic: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "connection_name": "list_None_test17102868504212522810_test_basic"} */

    select
      null as "database",
      tbl.table_name as name,
      tbl.table_schema as "schema",
      case when tbl.table_type = 'BASE TABLE' then 'table'
           when tbl.table_type = 'VIEW' and mv.table_name is null then 'view'
           when tbl.table_type = 'VIEW' and mv.table_name is not null then 'materialized_view'
           when tbl.table_type = 'SYSTEM VIEW' then 'system_view'
           else 'unknown' end as table_type
    from information_schema.tables tbl
    left join information_schema.materialized_views mv
    on tbl.TABLE_SCHEMA = mv.TABLE_SCHEMA
    and tbl.TABLE_NAME = mv.TABLE_NAME
    where tbl.table_schema = 'test17102868504212522810_test_basic'

23:40:51  SQL status: SUCCESS 4 in 0.0 seconds
23:40:51  On list_None_test17102868504212522810_test_basic: ROLLBACK
23:40:51  On list_None_test17102868504212522810_test_basic: Close
23:40:51  Using starrocks connection "master"
23:40:51  On master: BEGIN
23:40:51  Opening a new connection, currently in state init
23:40:51  SQL status: SUCCESS 0 in 0.0 seconds
23:40:51  On master: COMMIT
23:40:51  Using starrocks connection "master"
23:40:51  On master: COMMIT
23:40:51  SQL status: SUCCESS 0 in 0.0 seconds
23:40:51  On master: Close
23:40:51  Concurrency: 1 threads (target='default')
23:40:51  
23:40:51  Began running node snapshot.snapshot_strategy_timestamp.ts_snapshot
23:40:51  1 of 1 START snapshot test17102868504212522810_test_basic.ts_snapshot .......... [RUN]
23:40:51  Re-using an available connection from the pool (formerly list_None_test17102868504212522810_test_basic, now snapshot.snapshot_strategy_timestamp.ts_snapshot)
23:40:51  Began compiling node snapshot.snapshot_strategy_timestamp.ts_snapshot
23:40:51  Timing info for snapshot.snapshot_strategy_timestamp.ts_snapshot (compile): 16:40:51.852241 => 16:40:51.854489
23:40:51  Began executing node snapshot.snapshot_strategy_timestamp.ts_snapshot
23:40:51  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:51  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */
select distinct schema_name from information_schema.schemata
23:40:51  Opening a new connection, currently in state closed
23:40:51  SQL status: SUCCESS 8 in 0.0 seconds
23:40:51  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:51  On snapshot.snapshot_strategy_timestamp.ts_snapshot: BEGIN
23:40:51  SQL status: SUCCESS 0 in 0.0 seconds
23:40:51  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:51  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:51  SQL status: SUCCESS 7 in 0.0 seconds
23:40:51  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:51  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

  create table `test17102868504212522810_test_basic`.`ts_snapshot__dbt_tmp`
    PROPERTIES (
      "replication_num" = "1"
    )
  as with snapshot_query as (

    select * from `test17102868504212522810_test_basic`.`added`

    ),

    snapshotted_data as (

        select *,
            id as dbt_unique_key

        from `test17102868504212522810_test_basic`.`ts_snapshot`
        where dbt_valid_to is null

    ),

    insertions_source_data as (

        select
            *,
            id as dbt_unique_key,
            some_date as dbt_updated_at,
            some_date as dbt_valid_from,
            nullif(some_date, some_date) as dbt_valid_to,
            md5(concat_ws('|',coalesce(cast(id as char), '')
        , coalesce(cast(some_date as char), '')
        )) as dbt_scd_id

        from snapshot_query
    ),

    updates_source_data as (

        select
            *,
            id as dbt_unique_key,
            some_date as dbt_updated_at,
            some_date as dbt_valid_from,
            some_date as dbt_valid_to

        from snapshot_query
    ),

    insertions as (

        select
            'insert' as dbt_change_type,
            source_data.*

        from insertions_source_data as source_data
        left outer join snapshotted_data on snapshotted_data.dbt_unique_key = source_data.dbt_unique_key
        where snapshotted_data.dbt_unique_key is null
           or (
                snapshotted_data.dbt_unique_key is not null
            and (
                (snapshotted_data.dbt_valid_from < source_data.some_date)
            )
        )

    ),

    updates as (

        select
            'update' as dbt_change_type,
            source_data.*,
            snapshotted_data.dbt_scd_id

        from updates_source_data as source_data
        join snapshotted_data on snapshotted_data.dbt_unique_key = source_data.dbt_unique_key
        where (
            (snapshotted_data.dbt_valid_from < source_data.some_date)
        )
    )

    select * from insertions
    union all
    select * from updates

23:40:52  SQL status: SUCCESS 10 in 0.0 seconds
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot__dbt_tmp'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:52  SQL status: SUCCESS 9 in 0.0 seconds
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:52  SQL status: SUCCESS 7 in 0.0 seconds
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot__dbt_tmp'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:52  SQL status: SUCCESS 9 in 0.0 seconds
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:52  SQL status: SUCCESS 7 in 0.0 seconds
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

    select
        column_name,
        data_type,
        character_maximum_length,
        numeric_precision,
        numeric_scale

    from INFORMATION_SCHEMA.columns
    where table_name = 'ts_snapshot__dbt_tmp'

      and table_schema = 'test17102868504212522810_test_basic'

    order by ordinal_position

23:40:52  SQL status: SUCCESS 9 in 0.0 seconds
23:40:52  Writing runtime sql for node "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  Using starrocks connection "snapshot.snapshot_strategy_timestamp.ts_snapshot"
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "snapshot.snapshot_strategy_timestamp.ts_snapshot"} */

          update `test17102868504212522810_test_basic`.`ts_snapshot`, (select dbt_scd_id, dbt_change_type, dbt_valid_to from `test17102868504212522810_test_basic`.`ts_snapshot__dbt_tmp`) as DBT_INTERNAL_SOURCE
    set `test17102868504212522810_test_basic`.`ts_snapshot`.dbt_valid_to = DBT_INTERNAL_SOURCE.dbt_valid_to
    where DBT_INTERNAL_SOURCE.dbt_scd_id = `test17102868504212522810_test_basic`.`ts_snapshot`.dbt_scd_id
    and DBT_INTERNAL_SOURCE.dbt_change_type = 'update'
    and `test17102868504212522810_test_basic`.`ts_snapshot`.dbt_valid_to is null

23:40:52  starrocks adapter: StarRocks error: 1064 (HY000): Getting syntax error at line 3, column 68. Detail message: Unexpected input ',', the most similar input is {'SET'}.
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: ROLLBACK
23:40:52  Timing info for snapshot.snapshot_strategy_timestamp.ts_snapshot (execute): 16:40:51.855178 => 16:40:52.161686
23:40:52  On snapshot.snapshot_strategy_timestamp.ts_snapshot: Close
23:40:52  Database Error in snapshot ts_snapshot (snapshots/ts_snapshot.sql)
  1064 (HY000): Getting syntax error at line 3, column 68. Detail message: Unexpected input ',', the most similar input is {'SET'}.
  compiled Code at target/run/snapshot_strategy_timestamp/snapshots/ts_snapshot.sql
23:40:52  1 of 1 ERROR snapshotting test17102868504212522810_test_basic.ts_snapshot ...... [ERROR in 0.31s]
23:40:52  Finished running node snapshot.snapshot_strategy_timestamp.ts_snapshot
23:40:52  Using starrocks connection "master"
23:40:52  On master: BEGIN
23:40:52  Opening a new connection, currently in state closed
23:40:52  SQL status: SUCCESS 0 in 0.0 seconds
23:40:52  On master: COMMIT
23:40:52  Using starrocks connection "master"
23:40:52  On master: COMMIT
23:40:52  SQL status: SUCCESS 0 in 0.0 seconds
23:40:52  On master: Close
23:40:52  Connection 'master' was properly closed.
23:40:52  Connection 'snapshot.snapshot_strategy_timestamp.ts_snapshot' was properly closed.
23:40:52  
23:40:52  Finished running 1 snapshot in 0 hours 0 minutes and 0.38 seconds (0.38s).
23:40:52  Command end result
23:40:52  
23:40:52  Completed with 1 error and 0 warnings:
23:40:52  
23:40:52    Database Error in snapshot ts_snapshot (snapshots/ts_snapshot.sql)
  1064 (HY000): Getting syntax error at line 3, column 68. Detail message: Unexpected input ',', the most similar input is {'SET'}.
  compiled Code at target/run/snapshot_strategy_timestamp/snapshots/ts_snapshot.sql
23:40:52  
23:40:52  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
23:40:52  Command `cli snapshot` failed at 16:40:52.181837 after 0.82 seconds
23:40:52  Flushing usage events
--------------------------- Captured stdout teardown ---------------------------
23:40:52  Acquiring new starrocks connection '_test'
23:40:52  Dropping schema "schema: "test17102868504212522810_test_basic"
".
23:40:52  Using starrocks connection "_test"
23:40:52  On _test: BEGIN
23:40:52  Opening a new connection, currently in state init
23:40:52  SQL status: SUCCESS 0 in 0.0 seconds
23:40:52  Using starrocks connection "_test"
23:40:52  On _test: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "connection_name": "_test"} */

    drop schema if exists `test17102868504212522810_test_basic`

23:40:52  SQL status: SUCCESS 0 in 0.0 seconds
23:40:52  On _test: COMMIT
23:40:52  Using starrocks connection "_test"
23:40:52  On _test: COMMIT
23:40:52  SQL status: SUCCESS 0 in 0.0 seconds
23:40:52  On _test: Close
=========================== short test summary info ============================
FAILED tests/functional/adapter/test_basic.py::TestSnapshotCheckColsMyAdapter::test_snapshot_check_cols
FAILED tests/functional/adapter/test_basic.py::TestSnapshotTimestampMyAdapter::test_snapshot_timestamp
========================= 2 failed, 7 passed in 14.27s =========================
Finished running tests!
alberttwong commented 8 months ago

https://github.com/StarRocks/dbt-starrocks/issues/31

alberttwong commented 8 months ago

https://github.com/StarRocks/dbt-starrocks/issues/29

alberttwong commented 8 months ago

Right now we are bypassing 2 tests.