search

uber / fiber

Distributed Computing for AI Made Simple
https://uber.github.io/fiber/
Apache License 2.0
1.04k stars 108 forks source link

Azure Databricks Pyspark Support #4

Open roshmath opened 4 years ago

roshmath commented 4 years ago

Hi, Just wondering if this would work with resizing azure databrick clusters!

calio commented 4 years ago

Hi roshmath, Fiber doesn't support azure databrick clusters (yet). We want to hear from the community how much need there is for each of the platforms and prioritize our resources on the platform that are most requested.

At the same time, we also welcome contributions from the community. Adding a new backend for fiber isn't that hard, here is an example of how Kubernetes are supported in Fiber: https://github.com/uber/fiber/blob/master/fiber/kubernetes_backend.py

roshmath commented 4 years ago

Sounds great, thank you!