Closed ashwaninbs closed 1 year ago
Hi,
Have you deployed the Cloud Formation template provided in the blog post? It should deploy the nyctaxi database which seems to be missing in your environment (This database contains the source data used in the project).
Be careful to deploy the Cloud Formation template and your dbt project in us-east-1 region. The nyctaxi is a Glue database and cross region usage is not possible.
Yes. It worked after i installed CF template.
However, still I had error in step 1. I had to grant Lakeformation permissions to the role used in the solution on nyctaxi and dbt_nyc_metrics databases.
In addition, as you will know users cannot access Athena and run queries, unless they have a user who has been given permission in Lakeformation on the silver and gold tables.
However, in the end it works.
It will be helpful if you can create some instructions on how to run the solution on mac.
On a mac, you need to:
Then you can just install the adapter like this:
$ python3 -m venv dbt_venv
$ source dbt_venv/bin/activate
$ python3 -m pip install --upgrade pip
$ pip3 install --no-cache-dir --upgrade boto3
$ pip3 install --no-cache-dir dbt-core
$ pip3 install --no-cache-dir dbt-glue
Then you can play the blog starting from the chapter Clone the project
I am trying to follow this article https://aws.amazon.com/blogs/big-data/build-your-data-pipeline-in-your-aws-modern-data-platform-using-aws-lake-formation-aws-glue-and-dbt-core/
I ran dbt debug --profiles-dir profile successfully.
However, for the following i got error:
However, I ran into the following error:
dbt run --profiles-dir profile
Error is as follows: ( Can someone point me to the fix please)