search

Nike-Inc / brickflow

Pythonic Programming Framework to orchestrate jobs in Databricks Workflow
https://engineering.nike.com/brickflow/
Apache License 2.0
183 stars 39 forks source link

Certificate failed error while deploying the workflow to databricks #52

Closed amaldevknike closed 7 months ago

amaldevknike commented 11 months ago

I am facing certificate failed error while deploying the workflow to Databricks using below command from my Databricks cli terminal. "brickflow projects deploy --project hello-world-brick flow -e local" I followed the steps mentioned under Getting started section of GitHub(https://github.com/Nike-Inc/brickflow/tree/main). Steps from 1 to 6 are complete without any error. But while deploying the workflow using the above-mentioned command, I get an error as certificate validation failed. data bricks Token configuration with cli was successful and I am able to list the databricks-dbfs files from cli without any issues. but I am not sure why I get this error while deploying the workflow as I am able to complete all other steps as expected. I haven't used docker for the set up. Kindly advise what could be the possible resolution for this error. Please find below the screenshot of the error. image

amaldevknike commented 10 months ago

@asingamaneni Any guidance on this will be greatly appreciated.

newfront commented 10 months ago

This appears to be an issue with the validity of the GitHub ssl for the connection (via the screenshot).

HTTPSConnectionPool('GitHub.com', '443'): max retries for the /databricks/cli/releases/...version ...v0.203.0.

I vaguely remember setting up some certs for GitHub and it ends up redirecting to githubusercontent. Since the download redirects maybe it's an issue with local OS security.

See https://github.com/orgs/community/discussions/50963 and see if the troubleshooting helps you identify the cert issue. It could be a legit bug with GitHub!

asingamaneni commented 10 months ago

Please connect with network team or platform team for any additional help on this. This has to do with the Zscalar and having the certs available at the right location.