Support custom certificate truststores for EG dispatch to SSL Enabled Yarn

akshaykjain commented 4 years ago

Description

yarn-api-client Resource Manager consumes a verify flag that processes boolean values to verify the server's TLS certificate or a string that should be a path to a CA bundle to be used. There should be a way to support custom certificate truststore from enterprise_gateway for dispatch to SSL enabled Yarn.

Logs

status in jeg.log on running a Jupyter Notebook with custom cacerts not passed to Yarn RM:

[W 2020-09-24 15:36:52.281 EnterpriseGatewayApp] Query for kernel ID '50ab1833-65dc-463a-a6c8-7873e5386d18'
failed with exception: <class 'requests.exceptions.SSLError'> - 'HTTPSConnectionPool(host='cdhv632-master.fyre.ibm.com',
port=8090): Max retries exceeded with url: /ws/v1/cluster/apps?startedTimeBegin=1600986995000 (Caused by
SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))'.

Kernel failed to start:

Environment

Enterprise Gateway Version 2.1
Platform: CDH 6.0.1 with TLS enabled
Others: Truststores available for use

akshaykjain commented 4 years ago

Potential Fix https://github.com/jupyter/enterprise_gateway/pull/888

kevin-bates commented 4 years ago

Excellent - thank you @akshaykjain!

kevin-bates commented 4 years ago

Closed via https://github.com/jupyter/enterprise_gateway/pull/888

jupyter-server / enterprise_gateway