Support nested mapping for the process

[antho1404]

When using the mapping function of the process we can now only create inputs based on a flat data structure:

mapping:
  foo: ref or const
  bar: ref or const

It would be nice to be able to support arrays and nested data like in the services and provide a full mapping.

mapping:
  my:
    nested:
       object: ref or const
   array:
     - ref or const
     - ref or const

[NicolasMahe]

This could be done by simply adding a nested outputs in the oneof of value:

message Map {
      message Output {
        message Reference {
          // Key of the node in the graph. If empty, will be using the src of the edge.
          string nodeKey = 1 [
            (gogoproto.moretags) = 'hash:"name:1" validate:"required"'
          ];

          // Key of a specific parameter of the referenced node's output data.
          string key = 2 [
            (gogoproto.moretags) = 'hash:"name:2" validate:"required"'
          ];
        }

        // Key of the output.
        string key = 1 [
          (gogoproto.moretags) = 'hash:"name:1" validate:"required"'
        ];

        oneof value {
          // Input defined as reference.
          Reference ref = 2 [
            (gogoproto.moretags) = 'hash:"name:2" validate:"required_without=Constant"'
          ];
          mesg.protobuf.Value constant = 3 [
            (gogoproto.moretags) = 'hash:"name:3" validate:"required_without=Ref"'
          ];

+          // Nested outputs.
+          repeated Output outputs = 4 [
+            (gogoproto.moretags) = 'hash:"name:4" validate:"dive,required"'
+          ];
        }
      }

      // Key of the mapping.
      string key = 1 [
        (gogoproto.moretags) = 'hash:"name:1" validate:"required"'
      ];

      // Outputs of the mapping.
      repeated Output outputs = 2 [
        (gogoproto.moretags) = 'hash:"name:2" validate:"dive,required"'
      ];
    }

[NicolasMahe]

Also to test it, you can update on of the e2e service to accept nested data! that's part of the issue https://github.com/mesg-foundation/engine/issues/1446

[krhubert]

Just to let you know

message M {
  oneof value {
    repeated string s = 1;
  }
}  
// Compile error:
// Fields in oneofs must not have labels (required / optional / repeated).

Looking for a different solution, because the one with repeated Output won't compile.

Also, we need different representation for array and nested objects. so we need at least two fields.

[NicolasMahe]

@krhubert could you write here a quick update of the actual solution implemented?

[krhubert]

@NicolasMahe Sure :)

The solution was to create something similar to protobuf.Struct type, but with addition Reference as special Value. So now the output looks like:

       oneof value {
          Null null = 1 [
            (gogoproto.moretags) = 'hash:"name:1" validate:"required_without=Ref Constant List Map"'
          ];

          string string_const = 2 [
            (gogoproto.moretags) = 'hash:"name:2"'
          ];

          double double_const = 3 [
            (gogoproto.moretags) = 'hash:"name:3"'
          ];

          bool bool_const = 4 [
            (gogoproto.moretags) = 'hash:"name:4"'
          ];

          // Input defined as reference.
          Reference ref = 5 [
            (gogoproto.moretags) = 'hash:"name:5" validate:"required_without=Null Constant List Map"'
          ];

          // List represent array.
          List list = 6 [
            (gogoproto.moretags) = 'hash:"name:6" validate:"required_without=Null Ref Constant Map"'
          ];

          // Outputs represent a field.
          Map map = 7 [
            (gogoproto.moretags) = 'hash:"name:6" validate:"required_without=Null Ref Constant List"'
          ];
        }
      }

So this is almost 1:1 related with protobuf.Struct/Values but with Reference :)