Spark & Hadoop compatibility. - Githubissues

utdemir / distributed-dataset

A distributed data processing framework in Haskell.

BSD 3-Clause "New" or "Revised" License

114 stars 5 forks source link

Spark & Hadoop compatibility. #18

Open utdemir opened 5 years ago

utdemir commented 5 years ago

It makes sense to work on plaing nicely with Apache Spark & Hadoop ecosystem; so that people can start using distributed-dataset alongside with their existing data pipeline.

This is an umbrella issue to track the things we can do to achieve this:

YARN Backend: https://github.com/utdemir/distributed-dataset/issues/15
HDFS Support: https://github.com/utdemir/distributed-dataset/issues/16
Parquet Support: https://github.com/utdemir/distributed-dataset/issues/17