cube-js / cube

📊 Cube — The Semantic Layer for Building Data Applications
https://cube.dev
Other
17.73k stars 1.75k forks source link

How to connect azure cosmos db #648

Closed Gunasekaranv closed 1 week ago

Gunasekaranv commented 4 years ago

Hello, i am working on a project where i am using azure cosmos db for database queries, however i am not able to connect cubejs with azure cosmos db as datasource. Is there any community connector for azure cosmos db as it is not supported offically, if so i will really appreciate any help and if there isn't, are there any plans in future to provide a support?

paveltiunov commented 4 years ago

Hey @Gunasekaranv ! Thanks for posting this! I'm not aware of any community drivers for cosmo db and we don't have it on our road map as of right now. However we'd love this contribution! Here's more info on it: https://github.com/cube-js/cube.js/blob/master/CONTRIBUTING.md#implementing-driver

Gunasekaranv commented 4 years ago

Hi @paveltiunov , i written Cosmos driver

const CosmosClient = require("@azure/cosmos").CosmosClient;
const BaseDriver = require('@cubejs-backend/query-orchestrator/driver/BaseDriver');
const SqlString = require('sqlstring');
//const { promisify } = require('util');

const applyParams = (query, params) => {
  return SqlString.format(query, params);
};
class CosmosDriver extends BaseDriver {
  constructor(config) {
    super();

    this.config = {
      endpoint: process.env.CUBEJS_DB_HOST,
      key: process.env.CUBEJS_DB_KEY,
      databaseId: process.env.CUBEJS_DB_NAME,
      containerId: process.env.CUBEJS_DB_CONTAINERID,
      ...config
    };

    this.client = new CosmosClient(this.config);
    this.database = this.client.database(this.config.databaseId);
    this.container = this.database.container(this.config.containerId);
  }

  testConnection() {
    return this.query('SELECT *from c', []);
  }
  sleep(ms) {
    return new Promise((resolve) => {
      setTimeout(() => resolve(), ms);
    });
  }

  query(query, values) {
    try {
      console.log(query);
      const queryString = applyParams(
        query,
        (values || []).map(s => (typeof s === 'string' ? {
          toSqlString: () => SqlString.escape(s).replace(/\\\\([_%])/g, '\\$1').replace(/\\'/g, '\'\'')
        } : s))
      );
      while (true) {
        console.log(queryString);
        const queryExecution = this.container.items
          .query(queryString)
          .fetchAll();
        const status = queryExecution.QueryExecution.Status.State;
        if (status === 'FAILED') {
          throw new Error(queryExecution.QueryExecution.Status.StateChangeReason);
        }
        if (status === 'CANCELLED') {
          throw new Error('Query has been cancelled');
        }
        if (
          status === 'SUCCEEDED'
        ) {
          let results = this.container.items
            .query(queryString)
            .fetchAll();
          const [header, ...tableRows] = results.ResultSet.Rows;
          allRows.push(...(allRows.length ? results.ResultSet.Rows : tableRows));
          if (!columnInfo) {
            columnInfo = results.ResultSet.ResultSetMetadata.ColumnInfo
          }
          return allRows.map(r =>
            columnInfo
              .map((c, i) => ({ [c.Name]: r.Data[i].VarCharValue }))
              .reduce((a, b) => ({ ...a, ...b }), {})
          );
        }
        this.sleep(500);
      }
    } catch (err) {
      console.log(err.message);
    }
  }
}

module.exports = CosmosDriver;

but i am getting below error so please help me out,

Error:

Error while loading DB schema
TypeError: Cannot read property 'then' of undefined at CosmosDriver.tablesSchema (E:\Developements\cubejs\voca\node_modules\@cubejs-backend\cosmosdb-driver\node_modules\@cubejs-backend\query-orchestrator\driver\BaseDriver.js:74:29) at E:\Developements\cubejs\voca\node_modules\@cubejs-backend\server-core\core\DevServer.js:51:41 at processTicksAndRejections (internal/process/task_queues.js:97:5) at async E:\Developements\cubejs\voca\node_modules\@cubejs-backend\server-core\core\DevServer.js:29:9
lawrence999 commented 4 years ago

@Gunasekaranv Were you able to connect the CosmosDB with CubeJS?

kAutoman commented 1 year ago

Any updates regarding cosmosdb connector?

igorlukanin commented 1 year ago

@kAutoman I believe there're not any significant updates. Just in case you'd like to contribute, here are our driver contribution guidelines: https://github.com/cube-js/cube.js/blob/master/CONTRIBUTING.md#contributing-database-drivers

igorlukanin commented 1 week ago

For quite some time already, there's a third-party CosmosDB driver linked from the docs: https://cube.dev/docs/product/configuration/data-sources#third-party-drivers