search

twitter / summingbird

Streaming MapReduce with Scalding and Storm
https://twitter.com/summingbird
Apache License 2.0
2.14k stars 267 forks source link

Add some Execution support in scalding #674

Closed johnynek closed 8 years ago

johnynek commented 8 years ago

This allows you to convert a Producer[Scalding, T] to a Execution[TypedPipe[T]] given a DateRange and optional options.

This does not convert summingbird to using Execution internally, but it maybe we can side-step that if we can safely use it from Execution (for the time being).

johnynek commented 8 years ago

/cc @benpence can you take a look. I know @isnotinvain said you were thinking of related things.

johnynek commented 8 years ago

seems to pass tests for me now.

rubanm commented 8 years ago

👍