self-signed ssl certificate using WebHDFS

[snowch]

BigInsights on Cloud uses a self-signed ssl certificate for the WebHDFS https endpoint.

Does this streams connector allow me to specify the certificate that it should expect to receive from the WebHDFS endpoint so that I can prevent man-in-the-middle type attacks?

[alexpogue]

Yes. You can export your SSL certificate and add it to your truststore. Then you'll need to tell the connector where to find it using the keyStorePath and keyStorePassword parameters.

See step 4 in Using the toolkit to connect to HDFS in the BigInsights on Bluemix service:

4. SSL Configuration

By default, when connecting to HDFS on Bluemix, the server certificate is accepted without validation. If you decide to enable SSL certificate validation for the connection to Bluemix, complete the following steps.

• Follow these instructions to export the SSL certificate and add it to your truststore. * Note that you should export the certificate from the IBM BigInsights Home page, and not from the Ambari console. The BigInsights home page is typically at https://server_host_name:8443/gateway/default/BigInsightsWeb/index.html.
• Copy the truststore into a location accessible by your application directory, for example, /etc/mystore.jks.
• Set the keyStorePath and keyStorePassword parameters in your application to the path to your keystore and the keystore password respectively. For example:

     () as HDFS2FileSink_2 = HDFS2FileSink(In){
         param
          file : "output_file.txt" ;
          hdfsUri : "webhdfs://server_host_name:8443" ;
          keyStorePassword:"storepass";
          keyStorePath: "etc/store.jks";
          hdfsPassword: "bi_password";
          hdfsUser:"bi_user";
       }