[SUPPORT] Failed to deserialize HoodieAvroIndexRecord

[TheR1sing3un]

Tips before filing an issue

• Have you gone through our FAQs?

• Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

• If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced

When we try to serialize the HoodieAvroIndexedRecord and then deserialize back, it will cause failure like that:

A clear and concise description of the problem.

To Reproduce

Steps to reproduce the behavior:

1. Add test case in TestSerializationUtils

   

2. Run this test

Expected behavior

successful deserialization

Environment Description

• Hudi version :

• Spark version :

• Hive version :

• Hadoop version :

• Storage (HDFS/S3/GCS..) :

• Running on Docker? (yes/no) :

Additional context

Add any other context about the problem here.

Stacktrace

Add the stacktrace of the error.

[danny0405]

Should we serialize HoodieAvroIndexedRecord directly? It is only a wrapper for nested indexed record currently, we should unwrap it first then serialize.

[TheR1sing3un]

Should we serialize HoodieAvroIndexedRecord directly? It is only a wrapper for nested indexed record currently, we should unwrap it first then serialize.

Thanks for your reply~ Yes, there is no code logic to directly serialize the HoodieAvroIndexedRecord now, but for the sake of subsequent scalability, and its ser/deser related interfaces have been implemented, then I think it is necessary to ensure that this serialization logic can be successful.

[danny0405]

but for the sake of subsequent scalability

I didn't see, it's just a temporary in-memory wrapper adapter for hoodie record.

[TheR1sing3un]

but for the sake of subsequent scalability

I didn't see, it's just a temporary in-memory wrapper adapter for hoodie record.

If we're really not going to serialize it in the future, we can throw a HoodieNotSupportedException in ser/deser methods~ It doesn't seem reasonable that we leave a ser/deser methods that will always fail.

[danny0405]

If we're really not going to serialize it in the future, we can throw a HoodieNotSupportedException in ser/deser methods

yeah, can you confirm whether Flink can serialize it correctly? Do we have to always register those Avro SE/DE class or it is just specific for Spark?