Open ttim opened 6 years ago
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.
:white_check_mark: ttim
:x: Timur Abishev
Thinking back at this...
We could have an alternative approach: require OrderedSerialization everywhere we use Ordering, but have a low priority implicit that uses Kryo to supply the implicit if you have an ordering.
We're considering to enable
OrderedSerialization
for most users at Twitter. Currently we have a blocker for that - users needs to change a source code to enable it (and not only pass a parameter, which is much more beneficial for us because we have auto-tuning infrastructure to make incremental land of such things possible).I've tried to prototype how we can workaround this - basic idea is to introduce
KeyGrouping
class which holds implicitly providedOrdering
and generatedOrderedSerialization
. With this approach we can choose which one of them to use in runtime. Another pros of this approach:OrderedSerialization
if they want to use it - it would be provided by defaultOrderedSerialization
on top level you can accidentally break some other ordering use cases.Another thing which is nice about this approach is the fact it's source backward compatible.
This is just prototype (it's not even holds both
OrderedSerialization
andOrdering
, I used it to gatherOrdering
's stats across our repo) but it illustrates the idea.@johnynek what do you think?