[ibis] session set_execution_mode

[xmnlab]

todd
quick ask: I can set ibis (via pymapd) to execute in cpu mode via con.con._client.set_execution_mode(con.con.sessionid,2)
Can we add proper methods to set the execution mode for the session to both pymapd and ibis?

MikeH
@xmnlab, Todd is asking something different. The execution_mode cpu|gpu controls the execution in the engine, not how the results are returned.
For this (but not the ipc options), I think both uses are important for the API.
A mode setting on the connection
add the option to the ibis execute() . This is more difficult to implement because it’s a mode of the session, but better for the library to do that than the user.

todd 
:point_up_2:

todd 
yes both are important

todd 
sometimes i know that a full workflow is not going on the gpu, typically due to memory constraints, so id like the whole notebook/session to run on cpu

todd 
at other times there’s a specific query that i know is heavy and want to run on cpu

todd 
(and this is orthogonal to the ipc stuff, i generally don’t trust it/have found it slower, but i hear that may be changing with some of the stuff intel has been working on)

xmnlab
thanks @todd and @MikeH for the details. I will open an initial issue locally to track that ... and probably I will open another issue on ibis repo to open this discussion as it would change some client functions signatures

[xmnlab]

WIP: https://github.com/Quansight/ibis/tree/omniscidb-set-execution-mode

[xmnlab]

PR WIP: https://github.com/ibis-project/ibis/pull/2254

[xmnlab]

stopped due the sql-hint option that would be available in omniscidb soon.

[xmnlab]

sql-hint was implemented in v5.3.1 https://docs.omnisci.com/overview/release-notes#5-3-1-july-8-2020 this task is not blocked. just waiting for some documentation about this new feature.

[xmnlab]

some description about how to use it (from Venkat):

the simple thing here is to just attach the hint like this after the SELECT keyword, if the hint parameter is set

SELECT /*+ cpu_mode */ ... (edited) 

so i'd assume the iibis omnisci compiler sees the flag and then can add this. that's all is currently needed. if the Ibis compiler generats a common table expression (CTE) then you'd want to put it after the CTE expression like this
WITH prequery AS (SELECT blah blah blah) SELECT /*+ cpu_mode */ * FROM pre_query;

[xmnlab]

no progress since my last comment.