Investigate Firestore to Snowflake DAG Errors - Githubissues

astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

https://ask.astronomer.io/

Apache License 2.0

196 stars 47 forks source link

Investigate Firestore to Snowflake DAG Errors #274

Closed davidgxue closed 9 months ago

davidgxue commented 9 months ago

Describe the bug load_firestore_to_snowflake DAG keeps failing in prod https://clmkpsyfc000301kcjie00t1l.astronomer.run/do9voqis/dags/load_firestore_to_snowflake/grid?dag_run_id=scheduled__2024-01-15T00%3A00%3A00%2B00%3A00&task_id=load_request_data_from_firestore&tab=logs

To Reproduce Examples in LangSmith: https://smith.langchain.com/o/4942ae8b-e5be-4964-bb1f-7544886fa06d/projects/p/421b4053-56ab-4704-a77d-33c374e4eac1?columnVisibilityModel=%7B%22outputs%22%3Afalse%2C%22feedback_stats%22%3Atrue%2C%22reference_example%22%3Afalse%2C%22first_token_time%22%3Afalse%2C%22tags%22%3Afalse%2C%22metadata%22%3Afalse%7D&timeModel=%7B%22duration%22%3A%227d%22%7D&peek=accb37fa-6dc6-46f7-bf3c-79daf88e2d97

Expected behavior

Should not fail

davidgxue commented 9 months ago

Issue resolved. Root cause was incorrect GCP permission. Other failures are due to missing credentials fail. DAG now runs correctly in prod without failures.