hashicorp / nomad-spark

DEPRECATED: Apache Spark with native support for Nomad as a scheduler
44 stars 16 forks source link
nomad scheduler spark

DEPRECATED

The Nomad Spark integration is deprecated and will no longer be maintained by HashiCorp. Spark lacks a pluggable scheduler interface, necessitating this independent fork. Maintaining this fork requires significant effort to keep current with upstream Spark, persisting correct behavior of our integration while tracking Spark's constantly-advancing feature set. Ultimately, we felt that the level of effort did not justify the result: a second-class Spark experience inconsistent with HashiCorp standards.

The Nomad team continues to be excited about the future of our ecosystem. We’re currently focusing our efforts on building first-class experiences, with features such as autoscaling and integrations with Spinnaker and Airflow on the horizon. As always, feel free to reach out on our discussion forums at https://discuss.hashicorp.com/c/nomad.

The previous source is available on the master branch.