search

astronomer / ask-astro

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer
https://ask.astronomer.io/
Apache License 2.0
192 stars 47 forks source link

Add Exponential Backoff and Auto Retry in Fetch Page Content When Crawling #292

Closed davidgxue closed 7 months ago

davidgxue commented 8 months ago

As a result of this PR https://github.com/astronomer/ask-astro/pull/269 and this issue here https://github.com/astronomer/ask-astro/issues/262. It was previously resolved with adding a fixed wait time, but was not the ideal approach!

davidgxue commented 7 months ago

293 resolves this issue