search

cerndb / dist-keras

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.
http://joerihermans.com/work/distributed-keras/
GNU General Public License v3.0
623 stars 169 forks source link

how can I solve this issue? #34

Closed rayjang closed 6 years ago

rayjang commented 6 years ago

I really love your project and tried to use yours but I am in trouble like this. (on Ubuntu)I downloaded spark-2.2.0-hadoop2.7 and unzip. after that, I installed dist-keras. I faced this error. "# In Windows, ensure the Java child processes do not linger after Python has exited. Exception: Java gateway process exited before sending the driver its port number" at "sc = SparkContext(conf=conf)" I did googling but I can't find the way. Can you let me know how to solve it?

JoeriHermans commented 6 years ago

Could you post the bootstrapping code?

Joeri

rayjang commented 6 years ago

I use a 'workflow' ipython file you provided. image and I added this to .bashrc

image

only my $PATH is "/home/rayjang/anaconda3/bin:/bin:/usr/local/bin

JoeriHermans commented 6 years ago

Some Googling brought me to the possibility that PYSPARK_DRIVER_PYTHON etc might be the issue. Can you use a different Python? (E.g., Anaconda Python)

Let me know.

Joeri

rayjang commented 6 years ago

Thanks!! It was totally my fault. I didn't set up my hadoop conf. Sorry since I didn't read your instruction carefully

JoeriHermans commented 6 years ago

No problem :) If you have any other questions, feel free to post them.

Joeri