We're planning to roll out OrderedSerialization to more jobs. The initial plan was to add the RequiredBinaryComparators traits to our execution and job classes, but jobs could fail at runtime if TypedPipe transformations are done outside the job/execution body since the implicit macro wouldn't be available. For instance, we recommend people to extract transformations in companion objects for testability.
Solution
Introduce a mechanism that allows us to add the traits without failing at runtime if a transformation doesn't have the ordered serialization. By overriding requireOrderedSerializationMode, we can set the mode to Log, which will produce an error message at runtime.
Notes
I've enhanced the error message so users know how to fix the problem without having to ask us or look at the FAQ
I"ve kept backward compatibility, but I'm not sure if it's necessary
Problem
We're planning to roll out
OrderedSerialization
to more jobs. The initial plan was to add theRequiredBinaryComparators
traits to our execution and job classes, but jobs could fail at runtime ifTypedPipe
transformations are done outside the job/execution body since the implicit macro wouldn't be available. For instance, we recommend people to extract transformations in companion objects for testability.Solution
Introduce a mechanism that allows us to add the traits without failing at runtime if a transformation doesn't have the ordered serialization. By overriding
requireOrderedSerializationMode
, we can set the mode toLog
, which will produce an error message at runtime.Notes