search

Bridgewater / scala-notebook

Interactive Scala REPL in a browser
Other
741 stars 155 forks source link

How to import a class from a jar? #45

Open tribbloid opened 10 years ago

tribbloid commented 10 years ago

I'm trying to combine it with Apache Spark (one of the hottest framework for data analysis) but I don't know how to do it. Can someone give me a hint so I can possibly implement it myself? Of course, if someone is still maintaining this project.

MasseGuillaume commented 10 years ago

I think you only need to add it to the kernel dependencies https://github.com/Bridgewater/scala-notebook/blob/98eecbcf03a87c44df64b589bb455289235e9c71/project/build.scala#L101

tylerprete commented 10 years ago

Tribbloid, I'm trying to get it working with spark as well. Adding it to the kernel allows you to import it, but I'm still running into issues because scala notebook uses akka 2.1.4, while spark 1.0 uses akka 2.2.3. If you get it working, I'd love to hear about it, and I'll do the same.

tribbloid commented 10 years ago

Hi Tyler,

Thanks a lot! I'm not a proficient sbt user but I've encounter another jar hell problem in maven: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-throws-NoSuchFieldError-when-testing-on-cluster-mode-td8064.html#a8083

The solution in maven is to include maven-shade plugin and rename the akka package in the notebook (since spark-shell will always load himself first). I'll fork this repo and migrate to maven in a few days.

I'm not sure if sbt-assembly has a similar switch that can do this. Perhaps it worths a try :)

Yours Peng

KenCoder commented 10 years ago

I tried upgrading to akka 2.2 but had trouble with the remote forking. Maybe I'll try it again.

Ken

On Jun 29, 2014, at 9:10 PM, Tyler Prete notifications@github.com wrote:

Tribbloid, I'm trying to get it working with spark as well. Adding it to the kernel allows you to input it, but I'm still running it issues because scala notebook uses akka 2.1.4, while spark 1.0 uses akka 2.2.3. If you get it working, I'd love to hear about it, and I'll do the same.

— Reply to this email directly or view it on GitHub.

andypetrella commented 10 years ago

Hi all, I'm going to give this a try. However, I would prefer to catch up where you guyz are. Is there any sample repo from where I can go ahead? Thanks in advance

andypetrella commented 9 years ago

So I started an integration... the hard way, but works quite fine: https://github.com/andypetrella/scala-notebook/tree/spark

akhld commented 9 years ago

Any doc to get it working with Spark?

andypetrella commented 9 years ago

I'll try to put some doc in the README, and most probably a blog, however the changes are quite deep and will be hard (non trivial) to migrate back :-(

akhld commented 9 years ago

I gave this one a try https://github.com/andypetrella/scala-notebook/tree/spark but it couldn't compile the code. See the image below image

andypetrella commented 9 years ago

To not pollute this entry, could you fill an issue on the repo? Although this is not a compilation error but a (AFAICS) a misusage of sbt. There are several manner to use it, the longer but safer:

sbt project server run

The fastest:

sbt server/run