search

nightscape / spark-excel

A Spark plugin for reading and writing Excel files
Apache License 2.0
466 stars 146 forks source link

spark-excel not working with scala but working with Java #538

Open bmdoss opened 2 years ago

bmdoss commented 2 years ago

Capture

Hi, I tried spark-excel with Java it works fine,I mean basic Operations. but same it not working with Scala even I have tried with different version of scala, spark with spark-excel SAME ERROR please refer below.

Also attached sample screen shot(one of try).

Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: excel. Please find packages at http://spark.apache.org/third-party-projects.html

at org.apache.spark.sql.errors.QueryExecutionErrors$.failedToFindDataSourceError(QueryExecutionErrors.scala:443) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:670) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:720) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:210) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:188) at org.learnSpark.application.ReadTSV$.delayedEndpoint$org$learnSpark$application$ReadTSV$1(ReadTSV.scala:16) at org.learnSpark.application.ReadTSV$delayedInit$body.apply(ReadTSV.scala:5) at scala.Function0.apply$mcV$sp(Function0.scala:39) at scala.Function0.apply$mcV$sp$(Function0.scala:39) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17) at scala.App.$anonfun$main$1(App.scala:76) at scala.App.$anonfun$main$1$adapted(App.scala:76) at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563) at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561) at scala.collection.AbstractIterable.foreach(Iterable.scala:926) at scala.App.main(App.scala:76) at scala.App.main$(App.scala:74) at org.learnSpark.application.ReadTSV$.main(ReadTSV.scala:5) at org.learnSpark.application.ReadTSV.main(ReadTSV.scala) Caused by: java.lang.ClassNotFoundException: excel.DefaultSource at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:656) at scala.util.Try$.apply(Try.scala:210) at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:656) at scala.util.Failure.orElse(Try.scala:221) at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:656) ... 17 more 22/02/06 21:12:40 INFO SparkContext: Invoking stop() from shutdown hook

quanghgx commented 2 years ago

Hi @bmdoss If you don't mind, please help share (compiable & minimal) projects (java & scala) that can reproduce your described issue? Frankly, it looks more likely caused by two different setup (class path & dependencies) on your scala vs java builds. It's strange.