cube-js / cube

📊 Cube — The Semantic Layer for Building Data Applications
https://cube.dev
Other
17.91k stars 1.77k forks source link

Cassandra Driver: Create cubejs Hello world project with Cassandra #80

Open sameerkhan662 opened 5 years ago

sameerkhan662 commented 5 years ago

Describe the bug Permission denied to build using node-gyp

What I did affter successfully installing cli: cubejs create hello-world -d cassandra

**- Installing DB driver dependencies

> java@0.9.1 install /home/navs/CubeJS/hello-world/node_modules/java
> node-gyp rebuild

sh: 1: node-gyp: Permission denied
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! java@0.9.1 install: `node-gyp rebuild`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the java@0.9.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2019-04-16T12_10_36_327Z-debug.log

Cube.js Error ---------------------------------------

Error: npm install --save @cubejs-backend/jdbc-driver node-java-maven failed with exit code 1
    at ChildProcess.child.on.code (/root/.nvm/versions/node/v10.13.0/lib/node_modules/cubejs-cli/cubejsCli.js:31:16)
    at ChildProcess.emit (events.js:182:13)
    at maybeClose (internal/child_process.js:962:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5)**

Version: CubeJS version: 0.7.0

paveltiunov commented 5 years ago

Hey @sameerkhan662 ! Thanks for posting these. I guess there're two separate issues: cube.js lacks Cassandra driver and you have problems with node-java installation. Even if second issue is solved there will be still error because there's no Cassandra driver. Let's use this issue to track Cassandra driver development.

@sameerkhan662 if you have some spare time there's guide on how to implement it https://github.com/statsbotco/cube.js/blob/master/CONTRIBUTING.md#implementing-driver

sameerkhan662 commented 5 years ago

@paveltiunov Yes I have install nodejs as root then I switched to normal user and installed everything but Now I am getting something like

> node-gyp rebuild

gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: unable to get local issuer certificate
gyp ERR! stack     at TLSSocket.onConnectSecure (_tls_wrap.js:1049:34)
gyp ERR! stack     at TLSSocket.emit (events.js:182:13)
gyp ERR! stack     at TLSSocket._finishInit (_tls_wrap.js:631:8)
gyp ERR! System Linux 4.4.0-145-generic
gyp ERR! command "/home/navs/.nvm/versions/node/v10.13.0/bin/node" "/home/navs/.nvm/versions/node/v10.13.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/navs/hello-world/node_modules/java
gyp ERR! node -v v10.13.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! java@0.9.1 install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the java@0.9.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/navs/.npm/_logs/2019-04-16T14_03_46_770Z-debug.log

Cube.js Error ---------------------------------------

Error: npm install --save @cubejs-backend/jdbc-driver node-java-maven failed with exit code 1
    at ChildProcess.child.on.code (/home/navs/.nvm/versions/node/v10.13.0/lib/node_modules/cubejs-cli/cubejsCli.js:31:16)
    at ChildProcess.emit (events.js:182:13)
    at maybeClose (internal/child_process.js:962:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5)

Do you think I still have issue with cassandra driver though I have installed cassandra but have not started when I run the above code

paveltiunov commented 5 years ago

@sameerkhan662 Yep. Right. You're still having issue with Cassandra driver because there's no Cassandra driver at all. JDBC driver installed as fallback.

sameerkhan662 commented 5 years ago

I have not copied the folder structure yet but I am having a little confusion. Is it okay to copy the installed driver and paste in @cubejs-backend/<db-name>-driver. I doubt this thing if it will not break. Also mongo bi is not free therefore cubejs server can even connect to NoSQL db? Other npm issue is resolved now.

paveltiunov commented 5 years ago

@sameerkhan662 Yep. It's ok do that. It should be done in cloned cube.js git repository not installed module.

sameerkhan662 commented 5 years ago

I didn't understand will it work or not....also my driver structure is somewhat like this(in windows):

https://pasteboard.co/IgbfSbd.png

Should I copy this whole folder?

paveltiunov commented 5 years ago

@sameerkhan662 Ah. I see. Nope. You don't need to copy that. Added some clarifications here: https://github.com/statsbotco/cube.js/blob/master/CONTRIBUTING.md#implementing-jdbc-driver

sameerkhan662 commented 5 years ago

@paveltiunov I don't see anything which is related to NoSQL connection or just json based data structure(mongobi is not free). Could you please elaborate me how implementing-jdbc-driver is related to the above structure I've posted

paveltiunov commented 5 years ago

Hey @sameerkhan662 ! Yep. Sure! Cassandra itself is a NoSQL database. Cube.js works only with SQL backends. Fortunately there's JDBC SQL Driver for Cassandra which makes Cassandra a SQL backend: https://mvnrepository.com/artifact/org.apache-extras.cassandra-jdbc/cassandra-jdbc/1.2.1. It's actually CQL which is in some kind compliant with SQL: https://cassandra.apache.org/doc/latest/cql/index.html.

So in order to make Cassandra work with Cube.js we need:

  1. Implement JDBC connection driver: https://github.com/statsbotco/cube.js/blob/master/CONTRIBUTING.md#implementing-jdbc-driver
  2. Implement CQL dialect: https://github.com/statsbotco/cube.js/blob/master/CONTRIBUTING.md#implementing-sql-dialect
sameerkhan662 commented 5 years ago
const DbTypes = {
  cassandra: {
    driverClass: "com.mysql.jdbc.Driver",
    prepareConnectionQueries: [`SET time_zone = '+00:00'`],
    mavenDependency: {
      "groupId": "org.apache-extras.cassandra-jdbc",
      "artifactId": "cassandra-jdbc",
      "version": "1.2.1"
    },
    properties: {
      user: process.env.CUBEJS_DB_USER,
      password: process.env.CUBEJS_DB_PASS,
    },
    jdbcUrl: () => `jdbc:mysql://${process.env.CUBEJS_DB_HOST}:3306/${process.env.CUBEJS_DB_NAME}`
  }

First: What should be in place of driver class? Second: Where do I make these changes I am installing cubejs using command and these packages I am not able to find in my local drives since I don't even have JDBCDriver.js I have PostgresDriver.js in @cubejs-backend path. Third: Even I did this then what is meant by Find most similar BaseQuery implementation in @cubejs-backend/schema-compiler/adapter. Since I don't find any CassandraQuery.js therefore most similar to me seems to confuse me a little

paveltiunov commented 5 years ago

@sameerkhan662 Great questions! Most of info you can get from here: https://stackoverflow.com/questions/7688571/what-should-be-the-connection-string-while-using-cql-jdbc-driver

  1. You can try org.apache.cassandra.cql.jdbc.CassandraDriver.
  2. You should fork and clone https://github.com/statsbotco/cube.js repository for that.
  3. Yeah. It's a little bit tricky to select. I believe you can use MysqlQuery as a starting point.
CBox commented 4 years ago

const DbTypes = { cassandra: { driverClass: "com.mysql.jdbc.Driver", prepareConnectionQueries: [SET time_zone = '+00:00'], mavenDependency: { "groupId": "org.apache-extras.cassandra-jdbc", "artifactId": "cassandra-jdbc", "version": "1.2.1" }, properties: { user: process.env.CUBEJS_DB_USER, password: process.env.CUBEJS_DB_PASS, }, jdbcUrl: () =>jdbc:mysql://${process.env.CUBEJS_DB_HOST}:3306/${process.env.CUBEJS_DB_NAME}}

First: What should be in place of driver class? Second: Where do I make these changes I am installing cubejs using command and these packages I am not able to find in my local drives since I don't even have JDBCDriver.js I have PostgresDriver.js in @cubejs-backend path. Third: Even I did this then what is meant by Find most similar BaseQuery implementation in @cubejs-backend/schema-compiler/adapter. Since I don't find any CassandraQuery.js therefore most similar to me seems to confuse me a little

Did you had any success? I'd love to help with this!

paveltiunov commented 4 years ago

@CBox Hey Rafael! Please feel free to try it. I believe no one is on it currently.

CBox commented 4 years ago

@paveltiunov To be honest I found out I can connect to Spark SQL that is part of our stack so no need of Cassandra anymore :)

saulfrank commented 4 years ago

You could also look into using Prestodb as a connector.