search

astronomer / astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code
https://astronomer.github.io/astronomer-cosmos/
Apache License 2.0
774 stars 170 forks source link

Fix Netlify build timeouts #1294

Closed tatiana closed 3 weeks ago

tatiana commented 3 weeks ago

Netlify jobs are failing during the "Initialization" step as seen in: https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/67225eeb6eaeb400080afce5

With the error:

5:01:30 PM: Failed during stage 'Install dependencies': Command did not finish within the time limit
5:01:30 PM: Execution timed out after 17m59.815960796s
5:01:30 PM: Execution timed out after 17m59.917466353s
5:01:30 PM: Failing build: Failed to install dependencies

Dependency resolution is a pain. After confirming the dependencies that were taking longer, I pinned several of them, based on the constraints file: https://github.com/apache/airflow/blob/constraints-2.10.0/constraints-3.10.txt

This seems to have reduced the installation of the docs dependencies significantly:

Locally:

time pip install -r requirements.txt
42.67s

In Netlify, the Initialization step was reduced to less than 3 min: https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/672364e906837e00087c53ac

netlify[bot] commented 3 weeks ago

Deploy Preview for sunny-pastelito-5ecb04 ready!

Name Link
Latest commit b3cc801ce52c00bd4bd6fc56beb62d380f5be578
Latest deploy log https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/672364f985245d0008c87dbd
Deploy Preview https://deploy-preview-1294--sunny-pastelito-5ecb04.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.