Option for decoding with tagged union values

[LostKobrakai]

erlavro has :avro_decoder_hooks.tag_unions/0 to tag unions (e.g. of multiple records). It would be great if that hook could be enabled via some option when decoding. It seems for encoding tagged and non tagged values are supported already.

[Strech]

Hi @LostKobrakai do you have a specific example of when is it needed? I'm trying to keep the library small but functional, so some features definitely not 1:1 with erlavro

[LostKobrakai]

My current case is for permission handling, where a user can have only a single role, but each role has it's own set of additional metadata. I'd like to encode that in a single union of multiple different named records (name == role). That way I'm not encoding the role twice (e.g. in an enum + a separate union of records).

If you don't want to include specific union handling how about a generic way to customize the decoder hook used? This would be even more flexible.

[Strech]

Since I'm not eating my own dog food anymore, may I ask you to share a made-up example of data and schema that I can take a look at and see what is exactly happening? We can work around to see what could be done to achieve that.

[LostKobrakai]

That's the test I wrote to figure it out. Consider record_1 and record_2 to be role names, and each record can have role specific fields, which might or might not overlap. Not sure if this is actually a good way to compose hooks though.

test "decode with tagged tuple" do
    template = """
    {
      "type": "record",
      "name": "credit_wallet_payment_event",
      "namespace": "io.slim",
      "fields": [
        {
          "name": "record_union",
          "type": [
            {
              "type": "record",
              "name": "record_1",
              "fields": [
                {
                  "name": "start",
                  "type": "int"
                }
              ]
            },
            {
              "type": "record",
              "name": "record_2",
              "fields": [
                {
                  "name": "start",
                  "type": "int"
                }
              ]
            }
          ]
        }
      ]
    }
    """

    {:ok, schema} = Avrora.Schema.Encoder.from_json(template)

    data = %{record_union: {"io.slim.record_2", %{start: 12}}}

    {:ok, payload} = Avrora.Codec.Plain.encode(data, schema: schema)

    opts =
      Map.update!(Avrora.AvroDecoderOptions.options(), :hook, fn avrora_hook ->
        tagged_union_hook = :avro_decoder_hooks.tag_unions()

        fn type, sub_name_or_index, data, decode_fun ->
          tagged_union_hook.(type, sub_name_or_index, data, fn data ->
            avrora_hook.(type, sub_name_or_index, data, decode_fun)
          end)
        end
      end)

    assert %{"record_union" => {"io.slim.record_2", %{"start" => 12}}} ==
             :avro_binary_decoder.decode(payload, schema.full_name, schema.lookup_table, opts)

    assert {:ok, %{"record_union" => %{"start" => 12}}} =
             Avrora.Codec.Plain.decode(payload, schema: schema)
  end

[Strech]

Hey @LostKobrakai sorry for a long silence, have some IRL things, I will take a look on your example this week and try to share my thoughts

[Strech]

Evening @LostKobrakai I think I'm back to my nightlife and I've checked the erlavro and its tag union hook. Initially I have no interest to open access to the hook system (I don't have a good usage example), but with your case I think time comes and it makes sense to reconsider.

But instead of opening way to tag unions, I would rather open access to define custom hook which will be called inside of current Avrora.AvroDecoderOptions.__hook__/4 and result will be amended to handle null as nil (same as you did)

And any functionality could be injected (I'm also worried about debugging and will check how can I expose erlavro hooks traces. WDYT?

[Strech]

Thanks for your help, I really appreciate it, the new configuration option decoder_hook was released as a part of v0.24.0 release 🎉