Feature/support dbr 15.x - Githubissues

Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow

https://engineering.nike.com/brickflow/

Apache License 2.0

187 stars 44 forks source link

Feature/support dbr 15.x #161

Closed pariksheet closed 1 month ago

pariksheet commented 1 month ago

Update python dependancies to resolve conflict with DBR15.x also support DBR 11.x and above.

Description

System requirement for DBR15

System environment Operating System: Ubuntu 22.04.4 LTS

Java: Zulu 8.78.0.19-CA-linux64 Scala: 2.12.18 Python: 3.11.0 R: 4.3.2 Delta Lake: 3.2.0

Related Issue

Motivation and Context

How Has This Been Tested?

on databricks workflows using 15.4 LTS.

Screenshots (if appropriate):

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.