StarRocks / dbt-starrocks

dbt-starrocks contains all of the code enabling dbt to work with StarRocks
16 stars 6 forks source link

unsupported rename from table to view #28

Closed alberttwong closed 6 months ago

alberttwong commented 6 months ago
CLIENT: Server listening on port 49466...
Received JSON data in run script
Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=/Users/atwong/sandbox/dbt-starrocks', '/Users/atwong/sandbox/dbt-starrocks/tests/functional/adapter/test_basic.py::TestSimpleMaterializationsMyAdapter::test_base']
============================= test session starts ==============================
platform darwin -- Python 3.9.16, pytest-8.1.1, pluggy-1.4.0
rootdir: /Users/atwong/sandbox/dbt-starrocks
configfile: pytest.ini
plugins: dotenv-0.5.2
collected 1 item

tests/functional/adapter/test_basic.py F                                 [100%]

=================================== FAILURES ===================================
________________ TestSimpleMaterializationsMyAdapter.test_base _________________

self = <test_basic.TestSimpleMaterializationsMyAdapter object at 0x1190cbfa0>
project = <dbt.tests.fixtures.project.TestProjInfo object at 0x1070c85b0>

    def test_base(self, project):

        # seed command
        results = run_dbt(["seed"])
        # seed result length
        assert len(results) == 1

        # run command
        results = run_dbt()
        # run result length
        assert len(results) == 3

        # names exist in result nodes
        check_result_nodes_by_name(results, ["view_model", "table_model", "swappable"])

        # check relation types
        expected = {
            "base": "table",
            "view_model": "view",
            "table_model": "table",
            "swappable": "table",
        }
        check_relation_types(project.adapter, expected)

        # base table rowcount
        relation = relation_from_name(project.adapter, "base")
        result = project.run_sql(f"select count(*) as num_rows from {relation}", fetch="one")
        assert result[0] == 10

        # relations_equal
        check_relations_equal(project.adapter, ["base", "view_model", "table_model", "swappable"])

        # check relations in catalog
        catalog = run_dbt(["docs", "generate"])
        assert len(catalog.nodes) == 4
        assert len(catalog.sources) == 1

        # run_dbt changing materialized_var to view
        if project.test_config.get("require_full_refresh", False):  # required for BigQuery
            results = run_dbt(
                ["run", "--full-refresh", "-m", "swappable", "--vars", "materialized_var: view"]
            )
        else:
            results = run_dbt(["run", "-m", "swappable", "--vars", "materialized_var: view"])
        assert len(results) == 1

        # check relation types, swappable is view
        expected = {
            "base": "table",
            "view_model": "view",
            "table_model": "table",
            "swappable": "view",
        }
        check_relation_types(project.adapter, expected)

        # run_dbt changing materialized_var to incremental
>       results = run_dbt(["run", "-m", "swappable", "--vars", "materialized_var: incremental"])

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/adapter/basic/test_base.py:96: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = ['run', '-m', 'swappable', '--vars', 'materialized_var: incremental', '--project-dir', ...]
expect_pass = True

    def run_dbt(
        args: Optional[List[str]] = None,
        expect_pass: bool = True,
    ):
        # Ignore logbook warnings
        warnings.filterwarnings("ignore", category=DeprecationWarning, module="logbook")

        # reset global vars
        reset_metadata_vars()

        # The logger will complain about already being initialized if
        # we don't do this.
        log_manager.reset_handlers()
        if args is None:
            args = ["run"]

        print("\n\nInvoking dbt with {}".format(args))
        from dbt.flags import get_flags

        flags = get_flags()
        project_dir = getattr(flags, "PROJECT_DIR", None)
        profiles_dir = getattr(flags, "PROFILES_DIR", None)
        if project_dir and "--project-dir" not in args:
            args.extend(["--project-dir", project_dir])
        if profiles_dir and "--profiles-dir" not in args:
            args.extend(["--profiles-dir", profiles_dir])

        dbt = dbtRunner()
        res = dbt.invoke(args)

        # the exception is immediately raised to be caught in tests
        # using a pattern like `with pytest.raises(SomeException):`
        if res.exception is not None:
            raise res.exception

        if expect_pass is not None:
>           assert res.success == expect_pass, "dbt exit state did not match expected"
E           AssertionError: dbt exit state did not match expected

/Users/atwong/dbt-env/lib/python3.9/site-packages/dbt/tests/util.py:108: AssertionError
---------------------------- Captured stdout setup -----------------------------

=== Test project_root: /private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-79/project0
----------------------------- Captured stdout call -----------------------------

Invoking dbt with ['seed']
15:29:36  Running with dbt=1.6.2
15:29:36  Registered adapter: starrocks=1.4.2
15:29:36  Unable to do partial parsing because saved manifest not found. Starting full parse.
15:29:37  Found 3 models, 1 seed, 1 source, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
15:29:37  
15:29:37  Concurrency: 1 threads (target='default')
15:29:37  
15:29:37  1 of 1 START seed file test17102573770929885194_test_basic.base ................ [RUN]
15:29:37  1 of 1 OK loaded seed file test17102573770929885194_test_basic.base ............ [INSERT 10 in 0.25s]
15:29:37  
15:29:37  Finished running 1 seed in 0 hours 0 minutes and 0.34 seconds (0.34s).
15:29:37  
15:29:37  Completed successfully
15:29:37  
15:29:37  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Invoking dbt with ['run']
15:29:37  Running with dbt=1.6.2
15:29:37  Registered adapter: starrocks=1.4.2
15:29:37  Found 3 models, 1 seed, 1 source, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
15:29:37  
15:29:37  Concurrency: 1 threads (target='default')
15:29:37  
15:29:37  1 of 3 START sql table model test17102573770929885194_test_basic.swappable ..... [RUN]
15:29:37  1 of 3 OK created sql table model test17102573770929885194_test_basic.swappable  [SUCCESS 10 in 0.22s]
15:29:37  2 of 3 START sql table model test17102573770929885194_test_basic.table_model ... [RUN]
15:29:38  2 of 3 OK created sql table model test17102573770929885194_test_basic.table_model  [SUCCESS 10 in 0.20s]
15:29:38  3 of 3 START sql view model test17102573770929885194_test_basic.view_model ..... [RUN]
15:29:38  3 of 3 OK created sql view model test17102573770929885194_test_basic.view_model  [SUCCESS 0 in 0.04s]
15:29:38  
15:29:38  Finished running 2 table models, 1 view model in 0 hours 0 minutes and 0.50 seconds (0.50s).
15:29:38  
15:29:38  Completed successfully
15:29:38  
15:29:38  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Invoking dbt with ['docs', 'generate']
15:29:38  Running with dbt=1.6.2
15:29:38  Registered adapter: starrocks=1.4.2
15:29:38  Found 3 models, 1 seed, 1 source, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
15:29:38  
15:29:38  Concurrency: 1 threads (target='default')
15:29:38  
15:29:38  Building catalog
15:29:38  Catalog written to /private/var/folders/xn/_h8gfpfs3vv_m6gkxm8t0l380000gn/T/pytest-of-atwong/pytest-79/project0/target/catalog.json

Invoking dbt with ['run', '-m', 'swappable', '--vars', 'materialized_var: view']
15:29:38  Running with dbt=1.6.2
15:29:38  Registered adapter: starrocks=1.4.2
15:29:38  Unable to do partial parsing because config vars, config profile, or config target have changed
15:29:38  Found 3 models, 1 seed, 1 source, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
15:29:38  
15:29:38  Concurrency: 1 threads (target='default')
15:29:38  
15:29:38  1 of 1 START sql view model test17102573770929885194_test_basic.swappable ...... [RUN]
15:29:39  1 of 1 OK created sql view model test17102573770929885194_test_basic.swappable . [SUCCESS 0 in 0.05s]
15:29:39  
15:29:39  Finished running 1 view model in 0 hours 0 minutes and 0.11 seconds (0.11s).
15:29:39  
15:29:39  Completed successfully
15:29:39  
15:29:39  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Invoking dbt with ['run', '-m', 'swappable', '--vars', 'materialized_var: incremental']
15:29:39  Running with dbt=1.6.2
15:29:39  Registered adapter: starrocks=1.4.2
15:29:39  Unable to do partial parsing because config vars, config profile, or config target have changed
15:29:39  Found 3 models, 1 seed, 1 source, 0 exposures, 0 metrics, 335 macros, 0 groups, 0 semantic models
15:29:39  
15:29:39  Concurrency: 1 threads (target='default')
15:29:39  
15:29:39  1 of 1 START sql incremental model test17102573770929885194_test_basic.swappable  [RUN]
15:29:39  1 of 1 ERROR creating sql incremental model test17102573770929885194_test_basic.swappable  [ERROR in 0.22s]
15:29:39  
15:29:39  Finished running 1 incremental model in 0 hours 0 minutes and 0.28 seconds (0.28s).
15:29:39  
15:29:39  Completed with 1 error and 0 warnings:
15:29:39  
15:29:39    Compilation Error in macro rename_relation (macros/adapters/relation.sql)
  unsupported rename from table to view

  > in macro statement (macros/etc/statement.sql)
  > called by macro starrocks__rename_relation (macros/adapters/relation.sql)
  > called by macro rename_relation (macros/adapters/relation.sql)
  > called by macro materialization_incremental_default (macros/materializations/models/incremental/incremental.sql)
  > called by macro rename_relation (macros/adapters/relation.sql)
15:29:39  
15:29:39  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
=========================== short test summary info ============================
FAILED tests/functional/adapter/test_basic.py::TestSimpleMaterializationsMyAdapter::test_base
============================== 1 failed in 3.34s ===============================
Finished running tests!
Exception has occurred: CompilationError       (note: full exception trace is shown but execution is paused at: exception_handler)
Compilation Error in macro rename_relation (macros/adapters/relation.sql)
  unsupported rename from table to view

  > in macro statement (macros/etc/statement.sql)
  > called by macro starrocks__rename_relation (macros/adapters/relation.sql)
  > called by macro rename_relation (macros/adapters/relation.sql)
  > called by macro rename_relation (macros/adapters/relation.sql)
  File "/Users/atwong/sandbox/dbt-starrocks/dbt/adapters/starrocks/connections.py", line 176, in exception_handler (Current frame)
    yield
dbt.exceptions.CompilationError: Compilation Error in macro rename_relation (macros/adapters/relation.sql)
  unsupported rename from table to view
alberttwong commented 6 months ago

in mysql: https://github.com/dbeatty10/dbt-mysql/blob/cfd275c55adf229d890cd71460bcf520cda5166a/dbt/include/mysql/macros/adapters.sql#L59

alberttwong commented 6 months ago
20:30:18  On model.base.swappable: /* {"app": "dbt", "dbt_version": "1.6.2", "profile_name": "test", "target_name": "default", "node_id": "model.base.swappable"} */

  create table `test17102754100957863072_test_basic`.`swappable__dbt_tmp`
    PROPERTIES (
      "replication_num" = "1"
    )
  as 

  select * from `test17102754100957863072_test_basic`.`base`

20:30:18  SQL status: SUCCESS 10 in 0.0 seconds
20:30:18  starrocks adapter: Error running SQL: macro rename_relation
20:30:18  starrocks adapter: Rolling back transaction.
20:30:18  On model.base.swappable: ROLLBACK
20:30:30  Timing info for model.base.swappable (execute): 13:30:18.353744 => 13:30:30.514291
20:30:30  On model.base.swappable: Close
20:30:30  Compilation Error in macro rename_relation (macros/adapters/relation.sql)
  unsupported rename from table to view
alberttwong commented 6 months ago

https://github.com/StarRocks/dbt-starrocks/commit/6b6ae56e0231086538aaf8a57b8a5c80a4bbd6de