Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
187 stars 44 forks source link

[FEATURE] Optional auth for AutosysSensor #87

Closed maxim-mityutko closed 8 months ago

maxim-mityutko commented 9 months ago

Closes #86

Description

Making authentication optional for AutosysSensor. If airflow_cluster_auth is provided, it will be used to obtain the authentication token. The token then will be included in the request headers. Otherwise the call will happen without auth.

Related Issue

[FEATURE] Optional authentication for AutosysSensor

Motivation and Context

Skip authentication token acquisition if it's not required, reduces the amount of calls to the credentials store and / or authentication service, eg. Okta. Streamlines and simplifies user experience.

How Has This Been Tested?

Built wheel with the change, used wheel as a cluster library instead of the original brickflow distribution from PyPi. Executed workflow with AutosysSensor both with and without authentication. Both scenarios succeeded.

Screenshots (if appropriate):

Types of changes

Checklist:

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (548b6cb) 90.27% compared to head (9fc045f) 90.32%. Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #87 +/- ## ========================================== + Coverage 90.27% 90.32% +0.05% ========================================== Files 22 22 Lines 3361 3381 +20 ========================================== + Hits 3034 3054 +20 Misses 327 327 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

maxim-mityutko commented 8 months ago

@asingamaneni @stikkireddy can you please review this PR or let me if anything else should be changed / added?