twitter / scalding

A Scala API for Cascading
http://twitter.com/scalding
Apache License 2.0
3.49k stars 703 forks source link

Add Resolver type to clean up non-cascading backends #1835

Closed johnynek closed 6 years ago

johnynek commented 6 years ago

To break the pattern of making Sources and Sinks know about all the platforms, we need some way to look up the backend specific way to read/write. Much of that code will be the same, so I introduced a Resolver type to give us some basic combinators to work with (orElse, andThen, empty, pair being the most common).

This will also be used for the spark backend which is developing in #1832

johnynek commented 6 years ago

@ianoc @erik-stripe @fwbrasil can someone review?

ianoc commented 6 years ago

lgtm