Closed exalate-issue-sync[bot] closed 1 year ago
Navdeep commented: [~accountid:557058:afd6e9a4-1891-4845-98ea-b5d34a2bc42c] Don't totally understand this JIRA. It seems it is Sparklyr related, i.e., the port file
Navdeep commented: Next release will have the ability to point to a custom jar. So hopefully that helps this problem.
JIRA Issue Migration Info
Jira Issue: SW-300 Assignee: Navdeep Gill Reporter: Erin LeDell State: Resolved Fix Version: N/A Attachments: N/A Development PRs: N/A
JIRA Issue Migration Info Cont'd
Jira Issue Created Date: 2016-12-31T18:24:05.313-0800
Reported by a user:
We are contacting you because we are experiencing issues installing and executing rsparkling.
For security reasons, we have to work on a closed environment, therefore we cannot access and download packages via classic gradle or maven. We have downloaded and install rsparkling and all dependencies (sparklyr, sparkling-water…) manually.
Without this add-on our spark jobs runs just fine. But when using it, rsparkling tries to get some packages from distant repositories, which it cannot access.
When running the R command with rsparkling package attached : “sc <- spark_connect (master = "yarn-client")”, we obtain the following error:
{code} Error in start_shell(master = master, spark_home = spark_home, spark_version = version, :
Failed to launch Spark shell. Ports file does not exist.
Ivy Default Cache set to: /home/a454275/.ivy2/cache
The jars for the packages stored in: /home/a454275/.ivy2/jars
:: loading settings :: url = jar:file:/usr/hdp/2.4.2.0-258/spark/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar!/org/apache/ivy/core/settings/ivysettings.xml
ai.h2o#sparkling-water-core_2.10 added as a dependency
ai.h2o#sparkling-water-ml_2.10 added as a dependency
ai.h2o#sparkling-water-repl_2.10 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
You probably access the destination {code}