search

twitter / summingbird

Streaming MapReduce with Scalding and Storm
https://twitter.com/summingbird
Apache License 2.0
2.14k stars 267 forks source link

Storm platform fails in case of `sumByKey` in one branch and `flatMap` in another with `flatMap`ed source fails in runtime #725

Open ttim opened 7 years ago

ttim commented 7 years ago

Producer

val source = spoutSource.flatMap(e => List(e))
  source.sumByKey(store1).also(
    source.flatMap(branchFlatMap).sumByKey(store2)
  )

fails with java.lang.ClassCastException: java.lang.Integer cannot be cast to scala.Tuple2 exception in runtime.

johnynek commented 7 years ago

would be nice to fix this. Scary that something like this lasted this long.