search

dbt-msft / tsql-utils

dbt-utils for the dbt-msft family of packages
MIT License
25 stars 25 forks source link

Add ci #4

Closed dataders closed 3 years ago

alittlesliceoftom commented 3 years ago

Awesome work!

Can we also add instructions to run in local to the README.md?

alittlesliceoftom commented 3 years ago

It might help to have a consistent approach if some standard is defined.

alittlesliceoftom commented 3 years ago

no dbt_project.yml found at expected path /root/project/dbt-utils/dbt_project.yml

This error is actually same error i was getting when trying to run tests on local following the ci config. I assumed it was a clever CI setup thing.

dataders commented 3 years ago

Can we also add instructions to run in local to the README.md? yeah totally. if you can tell by my commits in this PR, I'm really no expert and am just throwing things at the wall to see what sticks...

The rough sketch of it is:

  1. open terminal in integration_tests/dbt_utils
  2. configure your ~/.dbt/profiles.yml to have an integration_tests section with a sqlserver target set as the default like integrations_tests/ci/sample.profiles.yml does
  3. run all the commands listed in the "Run Tests - dbt-utils" step of .circleci/config.yml namely: 
    dbt deps --target sqlserver
dbt seed --target sqlserver --full-refresh
dbt run --target sqlserver --full-refresh
dbt test --target sqlserver
dataders commented 3 years ago

no dbt_project.yml found at expected path /root/project/dbt-utils/dbt_project.yml

This error is actually same error i was getting when trying to run tests on local following the ci config. I assumed it was a clever CI setup thing.

same! what in the name of Linus Torvalds is going on here... It has something to do with how the integration_tests folder is somehow missing...

image
dataders commented 3 years ago

it has something to do with importing the dbt-utils repo as a submodule, something with which I'm very unfamiliar and will prolly have to lean on @jtcohen6 to help with...

UPDATE

figured it out @alittlesliceoftom sort of...

local set up steps

  1. open terminal in top-level dir of project
  2. run the following to install dbt-utils as a submodule git clone --recursive https://github.com/fishtown-analytics/dbt-utils
  3. open terminal in integration_tests/dbt_utils
  4. configure your ~/.dbt/profiles.yml to have an integration_tests section with a sqlserver target set as the default like integrations_tests/ci/sample.profiles.yml does
  5. run all the commands listed in the "Run Tests - dbt-utils" step of .circleci/config.yml namely: 
    dbt deps --target sqlserver
dbt seed --target sqlserver --full-refresh
dbt run --target sqlserver --full-refresh
dbt test --target sqlserver

Getting some errors with dbt seed now though

Database Error in seed data_date_trunc (data/cross_db/data_date_trunc.csv)
  ('23000', '[23000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Cannot insert an explicit value into a timestamp column. Use INSERT with a column list to exclude the timestamp column, or insert a DEFAULT into the timestamp column. (273) (SQLExecDirectW)')
dataders commented 3 years ago

closing in favor of #5