search

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.46k stars 2.43k forks source link

Hudi Record Key Data Type Must be String #6411

Closed sufei2009 closed 2 years ago

sufei2009 commented 2 years ago

I have a table where I set the record key in byte data type. It did not work until I changed it to string data type. Why does Hudi not allowing byte type for its record key? I use hudi on AWS with Lake Formation and Glue ETL for permission and cataloging

danny0405 commented 2 years ago

The byte primary key type expects to be supported, what exception it throws there for your use case ?

sufei2009 commented 2 years ago

We were not getting any exceptions. However, the partitions were acting inconsistent. Data was only writing to the last partition most of the time. Once we changed primary key type to string, the partitions worked correctly.

danny0405 commented 2 years ago

That's weird, we may need more details here.

nsivabalan commented 2 years ago

yes, can you furnish further details. any exceptions you see in driver logs. any reproducible scripts.

nsivabalan commented 2 years ago

@sufei2009 : gentle ping.

nsivabalan commented 2 years ago

if we don't hear back in a weeks time, we might close the issue due to inactivity. thanks!

xushiyan commented 2 years ago

closing due to inactivity