Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
183 stars 39 forks source link

[BUG] UcToSnowflakeOperator throws Incorrect Username and Password error #85

Closed rebecca-rajshree closed 7 months ago

rebecca-rajshree commented 7 months ago

Describe the bug UcToSnowflakeOperator throws Incorrect Username and Password error even though correct credentials are provided in databricks secrets. I assume the okta authentication is not being done when load_type is given as "full". Another thing I came across - Eventhough the load_type provided is "full", it still asks for the incremental_filter to be filled.

To Reproduce Steps to reproduce the behavior: Provide all necessary parameters for the UcToSnowflakeOperator along with authenticator and load_type = "full".

Expected behavior The authentication process should work when load_type = "full" and authenticator parameter is provided.

Screenshots

Cloud Information

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

ckonuganti commented 7 months ago

Looking into this issue