Closed bfejervari closed 4 years ago
adls:// uses the normal JVM proxy settings, including those for proxy auth. Start by getting coudstore to work with them, then go on to add them as params for distcp, which is troublesome but doable.
https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/bk_cloud-data-access/content/ch04s05.html
FWIW the storediag code was written to deal with adls proxy issues -the MSFT SDK doesn't check return content type on oauth authentication, and was trying to parse the corporate error html "use the proxy" as JSON and failing.
Hello, we want to connect a Cloudera Cluster to ADLS in a corporate environment. In the environment, every connection towards :443, :80 is blocked by central firewall - so any connection should use proxy, which has user+pw authenticated working method.
I used cloudstore-1.0.jar to test the connection and couldn't succeed.
I attach a simple script and the outputs of the run: cloudstore_ADLS_proxy_test.zip
The questions: how could I make cloudstore storediag (and any other hadoop tools) to connect to https://login.micrsoftonline.com throug an auth-enabled proxy?
Thanks in advance, best regards, Bence FEJÉRVÁRI