Snowflake-Labs / sfquickstarts

Follow along with our tutorials to get you up and running with Snowflake.
Apache License 2.0
343 stars 648 forks source link

[Accelerating Data Teams with dbt Core & Snowflake] multiple issues #223

Open cdgriffith opened 2 years ago

cdgriffith commented 2 years ago

https://quickstarts.snowflake.com/guide/data_teams_with_dbt_core/index.html has quite a few issues that have been blocking my progress, posting here to hopefully get these smoothed out for next poor soul trying it.

dbt Configuration

https://quickstarts.snowflake.com/guide/data_teams_with_dbt_core/index.html#2

Account name

No mention of how to get <your_snowflake_trial_account> with a trial account.

  1. You have to go to the app
  2. Click on the "Home" at top left
  3. Click on the random numbers and letters at bottom left
  4. Hover over your details
  5. Click on the small, light gray link symbol that when hovered over says "Copy Account URL"
  6. Use the full part (with several .s) between https://<>.snowflakecomputing.com snowflake

dbt init

Running dbt init now asks for info instead of modifying the file as suggested

22:52:24  Running with dbt=1.0.3
22:52:24  Creating dbt configuration folder at /home/james/.dbt
Which database would you like to use?
[1] snowflake

(Don't see the one you want? https://docs.getdbt.com/docs/available-adapters)

Enter a number: 1
account (https://<this_value>.snowflakecomputing.com): 
user (dev username): 
[1] password
[2] keypair
[3] sso
Desired authentication type option (enter a number): 1
password (dev password):
role (dev role): dbt_dev_role
warehouse (warehouse name): dbt_dev_wh
database (default database that dbt will build objects in): dbt_hol_dev
schema (default schema that dbt will build objects in): public
threads (1 or more) [1]: 200

Connect to Data Sources

https://quickstarts.snowflake.com/guide/data_teams_with_dbt_core/index.html?index=..%2F..index#4

DB name change

DB name change from KNOEMA_ECONOMY_DATA_ATLAS toECONOMY_DATA_ATLAS

Bad SQL select

SELECT * 
  FROM KNOEMA_ECONOMY_DATA_ATLAS.ECONOMY.USINDSSP2020
 WHERE "Date" = current_date();

doesn't work as we aren't in 2020 anymore

Building dbt Data Pipelines

https://quickstarts.snowflake.com/guide/data_teams_with_dbt_core/index.html?index=..%2F..index#5

Outdated and old repo deps

dbt deps

02:28:12  Running with dbt=1.0.3
  02:28:13  [WARNING]: Deprecated functionality
  The `fishtown-analytics/dbt_utils` package is deprecated in favor of
  `dbt-labs/dbt_utils`. Please update your `packages.yml` configuration to use
  `dbt-labs/dbt_utils` instead.
  02:28:14  Installing fishtown-analytics/dbt_utils
  02:28:14    Installed from version 0.6.4
  02:28:14    Updated version available: 0.7.0
  02:28:14
  02:28:14  Updates available for packages: ['fishtown-analytics/dbt_utils']
  Update your versions in packages.yml, then run dbt deps

Will later see the error

02:32:02  Running with dbt=1.0.3
02:32:02  Encountered an error:
Runtime Error
  Failed to read package: Runtime Error
    This version of dbt is not supported with the 'dbt_utils' package.
      Installed version of dbt: =1.0.3
      Required version of dbt for 'dbt_utils': ['>=0.18.0', '<0.20.0']
    Check for a different version of the 'dbt_utils' package, or run dbt again with --no-version-check

Updating to dbt-labs/dbt_util 0.8.1 fixed it

dbt pipelines - Stock trading history

https://quickstarts.snowflake.com/guide/data_teams_with_dbt_core/index.html?index=..%2F..index#6

Need table name updates

knoema_economy_data_atlas > economy_data_atlas

Column name change

02:33:27  Database Error in model base_knoema_stock_history (models/l10_staging/base_knoema_stock_history.sql)
02:33:27    000904 (42000): SQL compilation error: error line 6 at position 7
02:33:27    invalid identifier '"Company Symbol"'
02:33:27    compiled SQL at target/run/dbt_hol/models/l10_staging/base_knoema_stock_history.sql

Now "Company Symbol" changed to "Company Ticker", a lot of the page and images will need updated.

I haven't gone further yet, will update if I find more.

yarodmi commented 2 years ago

Great catch! Most of these would be covered with the update that we are about to merge. Thanks!