apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.35k stars 2.42k forks source link

[SUPPORT] Could you add Spark SQL Hint to be supported for hudi flexible query? #9075

Open zyclove opened 1 year ago

zyclove commented 1 year ago

Tips before filing an issue

Spark SQL Hint

/+ hoodie_prop('${tableName}', map('${key1}', '${value1}', '${key2}', ''${value2}'')) /


select
  /*+
    hoodie_prop(
      'default.h1',
      map('hoodie.datasource.read.begin.instanttime', '20221127083503537', 'hoodie.datasource.read.end.instanttime', '20221127083506081')
    ),
    hoodie_prop(
      'default.h2',
      map('hoodie.datasource.read.begin.instanttime', '20221127083508715', 'hoodie.datasource.read.end.instanttime', '20221127083511803')
    )
  */
  id, name, price, ts
from (
  select id, name, price, ts
  from default.h1
  union all
  select id, name, price, ts
  from default.h2
)

select /*+ hoodie_prop('default.h1', map('as.of.instant', '20221127102741536')) */)

# query mor  ro 
select /*+ hoodie_prop('default.h1', map('hoodie.datasource.query.type', 'read_optimized')) */ 

# query mor  rt 
select /*+ hoodie_prop('default.h1', map('hoodie.datasource.query.type', 'snapshot')) */

ref: https://www.modb.pro/db/570222

danny0405 commented 1 year ago

A cool feature, I think we can do this, put it to the release 1.0 roadmap.

ad1happy2go commented 1 year ago

Created JIRA for tracking for next major release - https://issues.apache.org/jira/browse/HUDI-6468

@zyclove In case you want to contribute here, feel free. we can help in case you need any support.

KnightChess commented 1 year ago

look like this case is from my blog, I Implementing this experimental function last year, but the community seems to be more inclined to tvf, you can try it. #7400 #8729 @zyclove