cube-js / cube

📊 Cube — The Semantic Layer for Building Data Applications
https://cube.dev
Other
17.93k stars 1.78k forks source link

Multi Query will cause create two duckdb driver #7581

Open wlf061 opened 10 months ago

wlf061 commented 10 months ago

Describe the bug I build a new image from cubejs/cube:v0.33.61, and fork duckdb driver:https://github.com/cube-js/cube/tree/v0.34.40/packages/cubejs-duckdb-driver, add log on the init function. When i send post query, the querytype is multi, There are two records about init,i think the dirver's constructor has called twice,and created to duckdb instance.

To Reproduce Steps to reproduce the behavior:

  1. Frok duckdb' project to repo and name it "customize-duckdb-cubejs-driver" , add log in constructor and init function image

  2. Build New Image:

`

  1. start cube service
  1. Send Multi Query { "query": { "measures": [ "test.currency", ], "dimensions": [ "test.local_book_date" ], "timeDimensions": [{ "dimension": "test.local_book_date", "compareDateRange": [["2023-06-01","2023-07-31"], ["2023-08-01","2043-12-11"]] } ] }, "queryType":"multi" }

The driver and db instance are init twice, the related log ` taihu-cube-pc-cube_api-1 | create duckdb driver taihu-cube-pc-cube_api-1 | create duckdb driver ..... taihu-cube-pc-cube_api-1 | init duckdb taihu-cube-pc-cube_api-1 | init duckdb

` Expected behavior The Driver and db are init once.

Version: v0.33.61 and v.34.40

igorlukanin commented 10 months ago

Hi @wlf061 👋 Let me add @ovr so he might provide his opinion as well. I think there are no strict guarantees on how many times a driver would be initialized. It might happen multiple times while Cube is running.

Do you think this is something undesirable?

wlf061 commented 10 months ago

@igorlukanin Thanks for your response. I think there has no problem,if init and release are performed in pairs. But from the code:

  1. init code ` public async getOrchestratorApi(context: RequestContext): Promise { .... this.orchestratorStorage.set(orchestratorId, orchestratorApi);

` we use default datasource and there has no configure of orchestrator, from the log, the orchestratorId are same for the multi query, Although created twice, only one instance is stored here.

  1. release execution chain `
    servers.ts:releaseConnections ->OrchestratorStorage.ts:releaseConnections->OrchestratorApi.ts:release->driver.release public async releaseConnections() { const result = [];

    for (const orchestratorApi of this.storage.values()) { result.push(orchestratorApi.release()); }

    await Promise.all(result);

    this.storage.clear(); }

` you can see, there has just release one driver.I think there has a leak. If there is something wrong with the analysis here, please let me know