dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
395 stars 221 forks source link

[Bug] CI is broken on `main` due to dependency resolution and timeout issues #979

Closed mikealfare closed 7 months ago

mikealfare commented 7 months ago

Is this a new bug in dbt-spark?

Current Behavior

CI for https://github.com/dbt-labs/dbt-spark/pull/977 is failing for code quality and unit tests due to timeout. The timeout is set at more than double the usual run. CI is also failing for integration tests. It times out at 90 minutes while still trying to resolve the dependencies:

pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000

Expected Behavior

Unit tests usually complete in a few minutes. Code quality usually completes in under a minute. Integration tests should take less than an hour I think. At very least, dependencies should be resolved in well under 90 minutes.

Steps To Reproduce

Run any of these CI checks against main:

Relevant log output

No response

Environment

- OS: ubuntu:latest
- Python: all
- dbt-core: `main` (1.8.0a1)
- dbt-spark: `main` (1.8.0a1)

Additional Context

I'm marking this is high priority because it potentially blocks merging into protected branches. However, this is not customer impacting, so I'm not marking it as High+ Severity.

martynydbt commented 7 months ago

This has already been fixed.