Closed irisschen closed 11 months ago
Thans for your report. Since in the view model, PipeRider uses the DuckDB client to connect directly to S3, S3 credentials are required.
Currently, PipeRider does not support S3 settings for dbt-duckdb profiles. However, you can pass credentials to the DuckDB client using S3 environment variables. Here's a example
export AWS_DEFAULT_REGION=ap-northeast-1
export AWS_ACCESS_KEY_ID=<id>
export AWS_SECRET_ACCESS_KEY=<secret>
piperider run
profiles.yml. If you export these envariable, dbt run
can still work
jaffle_shop:
target: dev
outputs:
dev:
type: duckdb
path: jaffle_shop.duckdb
extensions:
- httpfs
my source defintion
sources:
- name: external_source
tables:
- name: eorders
meta:
external_location: "read_csv('s3://piperider-athena-test/data/raw_orders.csv', AUTO_DETECT=TRUE)"
formatter: oldstyle
The view model
select * from {{ source('external_source', 'eorders') }}
It works after adding S3 environment variables, thanks!
Describe the bug Running into Error while executing
piperider run
in the view model with source file from an AWS S3 bucket .Screenshots
Desktop (please complete the following information):