SelectTransformer, used for codegen projections, attempts to optimize the projection expression to assign the columnar fields directly, in which case SelectTransformer populates SelectTransformationResult.ComputedFields, but there are cases where this is not possible, in which case SelectTransformer uses a less efficient fallback to use the projection expression wholesale and set via the StreamMessage indexer, populating SelectTransformationResult.ProjectionReturningResultInstance. The main bug is that many of the generated operators ignore ProjectionReturningResultInstance. The fix is to honor this case in the SelectTransformationResult - note that some operators expand this directly in the Transformer code (e.g., EquiJoin operators, GroupedWindow), while others defer to .tt file generation time (e.g., Select, SelectMany, Ungroup).
There is a secondary class of problems where the generated operators do not account for some expression when collecting reference assemblies for compilation. TemporalIngress does not accoutn for FuseModule expressions, and some EquiJoin operators did not account for the projection expressions. The fix is to account for these expressions when aggregating the reference assemblies.
There is also a minor fix to add an optimization to handle the case where the projection is a simple ValueTuple/Tuple creation, in which case we can bypass the creation and assign the columnar fields directly.
SelectTransformer, used for codegen projections, attempts to optimize the projection expression to assign the columnar fields directly, in which case SelectTransformer populates SelectTransformationResult.ComputedFields, but there are cases where this is not possible, in which case SelectTransformer uses a less efficient fallback to use the projection expression wholesale and set via the StreamMessage indexer, populating SelectTransformationResult.ProjectionReturningResultInstance. The main bug is that many of the generated operators ignore ProjectionReturningResultInstance. The fix is to honor this case in the SelectTransformationResult - note that some operators expand this directly in the Transformer code (e.g., EquiJoin operators, GroupedWindow), while others defer to .tt file generation time (e.g., Select, SelectMany, Ungroup).
There is a secondary class of problems where the generated operators do not account for some expression when collecting reference assemblies for compilation. TemporalIngress does not accoutn for FuseModule expressions, and some EquiJoin operators did not account for the projection expressions. The fix is to account for these expressions when aggregating the reference assemblies.
There is also a minor fix to add an optimization to handle the case where the projection is a simple ValueTuple/Tuple creation, in which case we can bypass the creation and assign the columnar fields directly.