search

twitter / scalding

A Scala API for Cascading
http://twitter.com/scalding
Apache License 2.0
3.49k stars 703 forks source link

Twitter release #1850

Closed fwbrasil closed 6 years ago

fwbrasil commented 6 years ago

@pankajroark @ttim @dieu @ktrivedisf @benpence could you review? I don't think the OSS tests will pass, but it should be ok since we've tested everything extensively internally

fwbrasil commented 6 years ago

@dieu these are actually changes already in develop. These are the commits that aren't in develop:

disable cross build - Necessary to make the internal release script work fix JobTest to use local mode - A port of a fix in develop fix hash joins to use multiple reducers - A port of a fix in develop expose flowstate - FlowState is now private[scalding] but we use it from scalding_internal. I'll send a PR as part of the next release I'll work on force to disk before making the iterable execution - Reverts toIterableExecution to the previous behavior so it forces the creation of a sink. This is what was making the e2e fail. I'll prepare a PR as part of the next release process

Sorry, I realized just now that I should have let you know this when I posted the PR