almond-sh / almond

A Scala kernel for Jupyter
https://almond.sh
BSD 3-Clause "New" or "Revised" License
1.59k stars 239 forks source link

Failed to import spark #1298

Closed grach0v closed 9 months ago

grach0v commented 9 months ago

I want to use spark in Jupyter Notebook. Scala work fine, but when I try to run import $ivy.`org.apache.spark::spark-sql:2.4.0` // Or use any other 2.x version here in a cell it gives me the following output

Downloading https://repo1.maven.org/maven2/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom
Downloaded https://repo1.maven.org/maven2/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom
Downloading https://repo1.maven.org/maven2/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom.sha1
Downloaded https://repo1.maven.org/maven2/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom.sha1
Downloading https://jitpack.io/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom
Downloaded https://jitpack.io/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom
Downloading https://jitpack.io/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom.sha1
Downloaded https://jitpack.io/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom.sha1
Failed to resolve ivy dependencies:Error downloading org.apache.spark:spark-sql_2.13:2.4.0
  not found: [/home/denis/.ivy2/local/org.apache.spark/spark-sql_2.13/2.4.0/ivys/ivy.xml](https://file+.vscode-resource.vscode-cdn.net/home/denis/.ivy2/local/org.apache.spark/spark-sql_2.13/2.4.0/ivys/ivy.xml)
  not found: https://repo1.maven.org/maven2/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom
  not found: https://jitpack.io/org/apache/spark/spark-sql_2.13/2.4.0/spark-sql_2.13-2.4.0.pom

How can I fix this error? Thanks for the help

dphung2000 commented 9 months ago

I believe it's gone, try version 3.5.0 instead of 2.4.0

grach0v commented 9 months ago

That helped Thank you! Maybe update docs https://almond.sh/docs/usage-spark ?

dphung2000 commented 9 months ago

they should revamp the doc for most of the parts, if not entirely. I struggled a bit with setting it up at first.