Instead of writing to DB and then calling in Fal py model. Can i run sql in fal model only and do write only once.

fal-ai / dbt-fal

do more with dbt. dbt-fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.

https://fal.ai/dbt-fal

Apache License 2.0

853 stars 74 forks source link

Instead of writing to DB and then calling in Fal py model. Can i run sql in fal model only and do write only once. #600

Open shivamsekra opened 2 years ago

shivamsekra commented 2 years ago

Context Instead of writing to DB and then calling in Fal py model. Can i run sql in fal model only and do write only once. Beacuse currently logic makes two writes which is not scalable

turbo1912 commented 2 years ago

Hi @shivamsekra happy to help with your request, but would like to understand your use case a little bit better. Are you just trying to transform a dbt model in place? dbt style data modelling usually recommends creating a new table for each stage of the transformation