sql-machine-learning / gomaxcompute

A Go driver for MaxCompute.
41 stars 11 forks source link

add query settings interface #46

Closed Yancey1989 closed 5 years ago

Yancey1989 commented 5 years ago

fixed #45

Yancey1989 commented 5 years ago

Thanks for @tonyyang-svail 's reminding, in the pyodps, there are two ways to set the query settings, the global one:

>>> from odps import options
>>> options.sql.settings = {'odps.sql.mapper.split.size': 16}
>>> o.execute_sql('select * from pyodps_iris')  # 会根据全局配置添加hints

and the one-off one:

o.execute_sql('select * from pyodps_iris', hints={'odps.sql.mapper.split.size': 16})

ref: https://pyodps.readthedocs.io/zh_CN/latest/base-sql.html#sql-hints

Maybe gomaxcompute need to implement both the two APIs. I think QueryWithHint is a good way to implement the one-off API, will implement it in this PR.

Yancey1989 commented 5 years ago

The script mode of Maxcompute on Aliyun: https://help.aliyun.com/document_detail/90082.html?spm=a2c4g.11186623.6.692.5036612drr5apU

tonyyang-svail commented 5 years ago

@Yancey1989 btw, do we need to support logging on a job?

Yancey1989 commented 5 years ago

@Yancey1989 btw, do we need to support logging on a job?

@tonyyang-svail , I think we need that, the logs message is useful such as the job view link.

Yancey1989 commented 5 years ago

Hi @tonyyang-svail , I file an issue to talk about the one-off API #50, maybe we can continue to talk about the API design there.

Yancey1989 commented 5 years ago

Hi @weiguoz @tonyyang-svail @typhoonzero I update this PR that define the query hints arguments in DSN:

access_id:access_key@service.com/api?curr_project=test_ci&scheme=http&hints_odps.sql.mapper.split_size=16

the URL query arguments key has prefix hints_ would be converted to query hints argument.

typhoonzero commented 5 years ago

Is this ready to merge? @Yancey1989