search

radanalyticsio / spark-operator

Operator for managing the Spark clusters on Kubernetes and OpenShift.
Apache License 2.0
156 stars 61 forks source link

For apps, add wget init containers #62

Open jkremser opened 6 years ago

jkremser commented 6 years ago

Currently, this works only for clusters.

jkremser commented 6 years ago

Could work also for fetching the jar files from maven central for a given GAV. Or fetching the wheel package from pypi once the pyspark is supported.

eosantigen commented 4 years ago

Guys, I went to use:

downloadData:
- url: https://bitbucket.org/metiscybertech/configuration-templates/raw/dac5298e5a83cfaa9ac07dc50f56cd255130faa5/core-site.xml
  to: /tmp/

When this is added all init containers fail into CrashLoopBackOff .

Any ideas? Thanks.

elmiko commented 4 years ago

hi @eosantigen

i am getting a 404 when i try to access that link. is it possible the link was removed or something?

edit: the links do need to be accessible from the container without credentials, just in case that wasn't clear.

also, i see the original title of this issue is about SparkApplication, is that the type of resource you are generating? if not, i think we need to open a new issue.

eosantigen commented 4 years ago

Hm, strange, for me it loads correctly. Have double and triple checked it. No 404 for me. I know that the links must be accessible without credentials. I am passing it in a SparkCluster type, as per the examples here https://github.com/radanalyticsio/spark-operator/blob/master/examples/with-prepared-data.yaml.

elmiko commented 4 years ago

ok, since your issue is about SparkCluster would you mind opening a new issue for us to track?

i just want to avoid mixing it with the SparkApplication use case that this issue was opened for.

eosantigen commented 4 years ago

ok will do thanks.