search

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
636 stars 116 forks source link

Hive support? #275

Closed concretevitamin closed 4 years ago

concretevitamin commented 5 years ago

Hi,

Thanks for the awesome work on this project. It seems to me by default Hive dependencies are not included, and there's no option to turn it on: https://github.com/nchammas/flintrock/blob/master/flintrock/services.py#L313.

Is there a workaround? Any chance this can get patched?

nchammas commented 5 years ago

Thank you for the kind words. I appreciate it.

That line is just for when you request Flintrock build Spark from source at a specific git commit. The release versions should all include Hive, I believe.

Are you trying to launch a Flintrock cluster with Spark built from source?

nchammas commented 4 years ago

Closing this due to its age. I believe the release versions of Spark (as opposed to the ones built from git) that Flintrock uses all have Hive included. If I'm mistaken, @concretevitamin, feel free to reopen this issue.