search

apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
https://gravitino.apache.org
Apache License 2.0
1.1k stars 352 forks source link

[Improvement] How to obtain the native type #2368

Open lw-yang opened 9 months ago

lw-yang commented 9 months ago

What would you like to be improved?

Some tables have many complex types, such as text and keyword types in es, and these field types will be mapped to the same java type string in gravitino. How to obtain the native type when reading

How should we improve?

Do we need to provide an additional attribute that keeps a record of the native field type

coolderli commented 9 months ago

This may be the same with https://github.com/datastrato/gravitino/issues/2373.

xiacongling commented 9 months ago

Native types may be important for engines to access data correctly. Same type in gravitino may have subtle differences when they come from different dbs/data warehouses. Engine connectors can do better work when knowing the details. IMO, gravitino should maintain and expose such imformation via its APIs.