search

Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
183 stars 36 forks source link

[FEATURE] Teams should not require to install libraries like airflow|snowflake|tableau to deploy brickflow_plugins #93

Open asingamaneni opened 6 months ago

asingamaneni commented 6 months ago

Is your feature request related to a problem? Please describe. Right now when we want to use any operators, by default users have to install apache-airflow, Tableau and Snowflake in their local virtual environment. Users should have to do it.

Cloud Information

Describe the solution you'd like We should validate the libraries available in bundle.yaml while deploying but not having them in our local venv.

Describe alternatives you've considered NA

Additional context Related to pr - #92

maxim-mityutko commented 6 months ago

I believe this is related: https://github.com/Nike-Inc/brickflow/issues/95