search

spark-notebook / spark-notebook

Interactive and Reactive Data Science using Scala and Spark.
Apache License 2.0
3.15k stars 653 forks source link

failed to run current build in powerShell #903

Open 108adams opened 7 years ago

108adams commented 7 years ago

Hello. Just for your information, I've got the same problem as in #893 when trying to start with powerShell, but it went ok in cmd. Anyway I had to add -Dconfig.file=conf\application.conf

Win10 home, JDK 8 update 131 Yours Adam

vidma commented 7 years ago

when trying to start with powerShell, but it went ok in cmd what do you mean?

markpavey commented 6 years ago

To get it to work on Windows I had to edit the file bin\spark-notebook.bat.

There is a line that sets the application classpath (line 132 in my build) and I had to add the conf directory to it:

set "APP_CLASSPATH=%CLASSPATH_OVERRIDES%;%YARN_CONF_DIR%;%HADOOP_CONF_DIR%;%EXTRA_CLASSPATH%;%APP_LIB_DIR%..\conf;%APP_CLASSPATH%"

I figured this out by comparing with the bin\spark-notebook bash script where the conf directory is already added to the classpath.

Is there a better way to do this or should this change be added to the build?

vidma commented 6 years ago

actually, you should always add explicit -Dconfig.file=conf\application.conf otherwise, sometimes it might sometimes behave weirdly, accidentally picking up wrong config file.

feel free to update docs if it's not clear enough somewhere.