search

cloudera-labs / envelope

Build configuration-driven ETL pipelines on Apache Spark
Apache License 2.0
158 stars 89 forks source link

Is there any plan to support Spark Structured Streaming? #37

Closed qiulin closed 5 years ago

qiulin commented 5 years ago

Is there any plan to support spark structured streaming ? Thank you for your awesome work.

jeremybeard commented 5 years ago

Yes, we do plan to move to Structured Streaming so that more of the functionality can be pushed down into Spark. We don't know yet when it will be released but it's a higher priority item.