search

IBMDataScience / DSx-Desktop

IBM Data Science Experience Desktop was built for those who want to download and play locally. Analyze, learn, and build with the tools you love, right on your desktop.
33 stars 18 forks source link

Accessing Cloudant/ApacheCouchDB doesn't work #30

Closed romeokienzler closed 7 years ago

romeokienzler commented 7 years ago

Accessing Cloudant from DSX Public works fine, but not from DSX Desktop, neither of the following three options works

Option 1 from DSX Documentation https://developer.ibm.com/clouddataservices/docs/ibm-data-science-experience/docs/load-and-filter-cloudant-data-with-spark/

Option 2-3 taken from my Stackoverflow Thread https://stackoverflow.com/questions/45207011/how-to-connect-to-cloudand-couchdb-using-sparksql-in-datascience-experience

romeokienzler commented 7 years ago

Played around a bit more and wanted to go for the bahir solution end ended up in JAR hell, please have a look at my notebook...

prashant182 commented 7 years ago

Hey Romeo, Thanks for pointing out your problem. I work with the dsx-desktop team, I can certainly help you with the issue.

I found couple of issues in your notebook, which is preventing you to connect with the Cloudant Database

  1. SPARK_CLASSPATH in your code is /usr/local/spark-2.1.0-bin-hadoop2.7/jars/ which is incorrect. please use $SPARK_HOME/jars instead.

  2. I noticed that you are using the latest version of playJson (2.6.2), unfortunately, it is not supported. Please use playJson(2.5.9), then you should be able to connect to CloudantDB.

I also have the working copy of the notebook you posted earlier, where I have corrected all these issues. Please let me know if you still have trouble connecting the DB I can send you that notebook so that you can better understand the solution.

romeokienzler commented 7 years ago

Hi, I've found the following solution to be working for me, can you please tell me if it is ok to do so? Otherwise I'll stick with yours...

romeokienzler commented 7 years ago

@prashant182 please append a GIST of the working copy, so that me and the community can benefit from it...

prashant182 commented 7 years ago

Hey Romeo, I saw the solution you posted earlier. I still think that using /usr/local/spark-2.1.0-bin-hadoop2.7/ instead of $SPARK_HOME is incorrect as we have a different spark path in our environment. Anyways if it is working for you, I am glad it worked out. Please take a look at the code here and let me know.