twitter / summingbird

Streaming MapReduce with Scalding and Storm
https://twitter.com/summingbird
Apache License 2.0
2.14k stars 267 forks source link

WaitingState.begin should perhaps return Try #650

Open johnynek opened 8 years ago

johnynek commented 8 years ago

https://github.com/twitter/summingbird/blob/develop/summingbird-batch-hadoop/src/main/scala/com/twitter/summingbird/batch/JobState.scala#L25

There are reasonable implementations that might not always be able to request a next interval (imagine state passed in from the args, we can make one request, but not the next). Now there is no way to do this without exceptions or null.

Since the ScaldingPlatform just returns the WaitingState from RunningState.succeed, that second one can be null or can throw on .prepare, and it seems things will work, just be a bit ugly.