search

CODAIT / r4ml

Scalable R for Machine Learning
Apache License 2.0
42 stars 13 forks source link

Ability to run R4ML from within Apache Livy using Apache Zeppelin #57

Closed gss2002 closed 6 years ago

gss2002 commented 6 years ago

I am wondering from my testing R4ML does not seem to work inside of Zeppelin while using Apache Livy as it seems like r4ml.session() attempts to build an entire new SparkR session. Any thoughts?

bdwyer2 commented 6 years ago

Hi gss2002, R4ML requires that the SystemML jar file be attached to your Spark session, thats why R4ML attempts to create its own Spark session. Are you able to pass in jar files to Livy when it starts?

gss2002 commented 6 years ago

@bdwyer2 let me give that a shot. That make sense let me try to pass it in

bdwyer2 commented 6 years ago

@gss2002 let me know if it works for you. You might have to set r4ml.env$R4ML_SESSION_EXISTS to TRUE to trick R4ML into running. Keep me posted!

gss2002 commented 6 years ago

Doesn't work... As if the session exists it cannot execute the init phase..

bdwyer2 commented 6 years ago

Can you please share the code you ran and the error message you received?