proposal: any type handling

[vbarua]

This PR is split into two commits:

• The first commits adds code for handling google.protobuf.Any types in the AdvanceExtension message, ReadRel.ExtensionTable message and ExtensionSingleRel messages.
• The second commit includes tests for the ProtoRelConverter, including a demonstration of how it would be overridden by the user to handle google.protobuf.Any types.

Summary

com.protobuf.Any types are handled by methods in the ProtoRelConverter that are intended to be overriden by end users. For example:

public class ProtoRelConverter {
  ...  
  protected Extension.Optimization optimizationFromAdvancedExtension(com.google.protobuf.Any any)
  ...
}

can be overriden by end users to convert the Any into their own Java equivalent.

Extension.Optimization is a tag interface to be able to distinguish between various types of Any data (instead of having Object everywhere).

/** Contains tag interfaces for handling {@link com.google.protobuf.Any} types within Substrait. */
public class Extension {

  private interface ToProto {
    com.google.protobuf.Any toProto();
  }

  public interface Optimization extends ToProto {}
 ...

[JamesRTaylor]

This is really cool! Thanks so much for putting it together, @vbarua. What do you think would be a good real world use case for this feature?

[vbarua]

@JamesRTaylor

What do you think would be a good real world use case for this feature?

Well, some of the stuff we want to use it for:

• ExtensionTable: We have a couple of internal systems we fetch data from. Our planner system is seperate from our execution system, and we want to use Extension Tables to embed query messages into the ReadRel. Effectively, our planner can already convert Substrait to our internal query messages, but our executor can't and we'd prefer not to duplicate the logic.
• ExtensionSingleRel: We have an in-house executor which has some custom physical relations, so we need to be able to model them in Substrait to use them.
• AdvancedExtensions: We're looking at adding some custom metadata for debug purposes to ensure that the plans generated by our planner don't violate invariants in our executor.

[vbarua]

Superceded by https://github.com/substrait-io/substrait-java/pull/135