Uncaught TypeError: $(...).highcharts is not a function.

weand commented 8 years ago

First: Much appreciation for this library!

Using the pure spark-highcharts library doesn't load the highcharts.js File inline. Web browser's console says somehting like: Uncaught TypeError: $(...).highcharts is not a function.

Question: Do I really need a custom zeppelin-web application to get spark-highcharts running?

After loading the highcharts file manually (by hacking the following snippet in the browser's console), diagrams are plotted sucessfully.

jQuery.getScript('https://code.highcharts.com/highcharts.js', function() {
      jQuery.getScript('https://code.highcharts.com/modules/exporting.js', function() {
})})

Is there a chance to provide an API for loading the js-library inline? Something that produces the following:

jQuery.when(
    jQuery.getScript('https://code.highcharts.com/highcharts.js', function() {
      jQuery.getScript('https://code.highcharts.com/modules/exporting.js', function() {
        jQuery("#tschart_${paragraphId}").ready,
        jQuery.Deferred(function( deferred ){
            jQuery( deferred.resolve );
        })      
      })
    })
).done(function(){
 ... highcharts code goes here
})

rockie-yang commented 8 years ago

Thanks very much for trying out spark-highcharts.

There are few ways which can add highcharts to a notebook.

Customize Zeppelin, You can do similar with I did for zeppelin-highcharts docker image.
Using %angular in one paragraph, In this case every notebook need do this.

weand commented 8 years ago

Thanks for your answer. Didn't recognize the zeppelin documtation part.

I think the dependency part is a bit outdated, and has a classloader leak. Are you aware of that the whole dependency tree (scope=compile) is loaded into the spark classpath, which means: 1) explicit dependency to net.liftweb:lift-json_2.11:2.6.3 can be omited 2) com.knockdata:spark-highcharts:0.6.1 is the only dependency required. 3) all nested compile dependencies will also be added.

the following jars are then part of the 'zeppelin application' within spark:

commons-csv-1.1.jar
scalap-2.11.1.jar
scala-reflect-2.11.1.jar
univocity-parsers-1.5.1.jar
scala-parser-combinators_2.11-1.0.1.jar
scala-xml_2.11-1.0.2.jar
lift-json_2.11-2.6.3.jar
zeppelin-spark_2.11-0.6.1.jar
spark-csv_2.11-1.4.0.jar
spark-highcharts-0.6.1.jar
scala-compiler-2.11.1.jar
paranamer-2.4.1.jar
scala-library-2.11.7.jar

At least the following jars already exist in the spark classpath (see SPARK_HOME/jars)

scala-compiler-2.11.8.jar
scala-library-2.11.8.jar
scalap-2.11.8.jar
scala-parser-combinators_2.11-1.0.4.jar
scala-reflect-2.11.8.jar
scala-xml_2.11-1.0.2.jar

Now the leak is: Having two version of the same library in classpath is no good idea. This may lead to weird classloader issues.

Solution 1) setting excludes for dependency 2) in zeppelin: org.scala-lang:scala-library,org.scala-lang:scalap,org.scala-lang:scala-compiler

This results in low level issues of the zeppelin spark interpreter, therefor no good idea:

ERROR [2016-09-27 14:22:49,210] ({pool-2-thread-2} Job.java[run]:189) - Job failed
java.lang.NoSuchMethodError: scala.reflect.internal.settings.MutableSettings$SettingValue.valueSetByUser()Lscala/Option;
        at scala.tools.nsc.Global.<init>(Global.scala:334)
        at scala.tools.nsc.interpreter.IMain$$anon$1.<init>(IMain.scala:247)
        at scala.tools.nsc.interpreter.IMain.newCompiler(IMain.scala:247)
        at scala.tools.nsc.interpreter.IMain.<init>(IMain.scala:93)
        at scala.tools.nsc.interpreter.IMain.<init>(IMain.scala:113)
        at scala.tools.nsc.interpreter.ILoop$ILoopInterpreter.<init>(ILoop.scala:108)
        at scala.tools.nsc.interpreter.ILoop.createInterpreter(ILoop.scala:118)
        at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:698)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
        at org.apache.zeppelin.spark.PySparkInterpreter.getSparkInterpreter(PySparkInterpreter.java:504)
        at org.apache.zeppelin.spark.PySparkInterpreter.createGatewayServerAndStartScript(PySparkInterpreter.java:180)
        at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:158)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
        at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Solution 2) Add excludes to pom.xml

...
   <dependency>
      <groupId>com.databricks</groupId>
      <artifactId>spark-csv_${scala.binary.version}</artifactId>
      <version>1.4.0</version>
      <exclusions>
          <exclusion>
              <groupId>org.scala-lang</groupId>
              <artifactId>scala-library</artifactId>
          </exclusion>
      </exclusions>
    </dependency>
...
    <dependency>
      <groupId>net.liftweb</groupId>
      <artifactId>lift-json_${scala.binary.version}</artifactId>
      <version>2.6.3</version>
      <exclusions>
          <exclusion>
              <groupId>org.scala-lang</groupId>
              <artifactId>scala-library</artifactId>
          </exclusion>
          <exclusion>
              <groupId>org.scala-lang</groupId>
              <artifactId>scalap</artifactId>
          </exclusion>
      </exclusions>
    </dependency>
...

Should I submit a PR for Solution 2) ?

rockie-yang commented 8 years ago

You are absolutely right. The document was dated.

It will be great if you can create a PR for Solution 2.

BTW, the zeppelin-highcharts docker image is I used to verify if the system is working.

weand commented 8 years ago

done. before submitting this:

According to your documentation only spark-highcharts and lift-json_2.11-2.6.3.jar needs to be in classpath. Can you confirm, that these can also be excluded:

commons-csv-1.1.jar
univocity-parsers-1.5.1.jar
spark-csv_2.11-1.4.0.jar
paranamer-2.4.1.jar

They derive from the dependency tree mentioned above.

rockie-yang commented 8 years ago

spark-csv shall be scope test since it's only use for test case. lift-json perhaps shall be provided, what do you think?

weand commented 8 years ago

since there are imports for net.liftweb.json packages in the model, following imports are required.

lift-json_2.11-2.6.3.jar
paranamer-2.4.1.jar (nested dependency from lift-json)

adapted changes in new PR. Can you submit and release a new version to maven central?

rockie-yang commented 8 years ago

I have uploaded the version 0.6.2 to maven central. And update the docker image to 0.6.2.

Could you please help to verify?

weand commented 8 years ago

Verified usage within existing zeppelin installation. +1

Didn't check the docker image.

rockie-yang commented 8 years ago

Thanks @weand

I was testing with the docker image. It turned out I have wrong configuration messed up with master branch and latest image.

I have fixed it.

Shall we close this issue?

knockdata / spark-highcharts

Uncaught TypeError: $(...).highcharts is not a function. #12