BigTable API

[callmehiphop]

Using this issue to try and get the BigTable API rolling, I started a rough draft and received some great initial feedback from @stephenplusplus, now I'm hoping to pull in more thoughts/criticisms/etc.

/cc @jgeewax

[jgeewax]

Ideally, it'd be super nice if we could use one of the existing libraries and swap out the transport layer. For example, in Python, we're trying to make it be something like:

from gcloud.bigtable import happybase

which would then act like the happybase library but send requests to Cloud Bigtable.

If we could do similarly with Node, that'd be really cool. Take a look at https://github.com/GoogleCloudPlatform/gcloud-node/issues/707 also

[callmehiphop]

Going to throw this out here to avoid spamming everyone in #707. It doesn't really seem like hbase is very popular in the node world (some one correct me if that's wrong), however the BigTable variant LevelDB seems fairly popular.

There's a LevelDB node library called levelup that allows you to easily swap out transport layers and even offers an abstract class to help create your own. While BigTable and LevelDB aren't really 1:1, I feel like this might be something worth looking into.

Code could maybe look something like..

var levelup = require('levelup');

var db = levelup('my-table', {
  db: require('gcloud-node-bigtable/leveldown'),
  projectId: 'grape-spaceship-123',
  keyFilename: '/path/to/keyfile.json'
});

Thoughts..?

edit: for stuff like zones/clusters/table admin we could require gcloud-node-bigtable directly and offer APIs specifically for that - this would be nice for helping us manage the different clients as well.

var bigtable = require('gcloud-node-bigtable')({
  projectId: 'grape-spaceship-123',
  keyFilename: '/path/to/keyfile.json'
});

var myZone = bigtable.zone('my-zone');

myZone.createCluster({ ... }, function (err, cluster, apiResponse) {});

[jgeewax]

@maxluebbe

The goal here would be....

1) have our own Bigtable API that people can use if they are new to Bigtable/hbase

2) provide adapters for gcloud node so that people who use existing hbase libraries can change an import and some config, and their existing code will work (for example, in Java, we just have a separate jar you pull in for Bigtable, but the api methods in code are the same)

Not sure how popular leveldb is and if we will be attracting the right userbase with it, but worth exploring.

[stephenplusplus]

I think @callmehiphop's idea is :ok_hand:. Our alternative is hacking over the pre-stable, CoffeeScript lib https://github.com/falsecz/hbase-rpc-client.

Not sure how popular leveldb is

Let's let npm stats do the talking:

Downloads in the last week:

hbase-rpc-client

• 7 downloads in the last day
• 27 downloads in the last week

levelup

• 3,016 downloads in the last day
• 14,852 downloads in the last week

...and if we will be attracting the right userbase with [leveldb], but worth exploring.

I don't know about this, but it will at least be an API Node devs could have used before.

Now that grpc can be installed with only Node dependencies, we shouldn't have to spawn this Bigtable stuff off into a separate module. @callmehiphop with your plan, will our library wrap level(up|down) and provide a complete solution for doing lookups, puts, etc? Or do we just offer an adapter that can be plugged into level(up|down) which the user has to manually install?

[callmehiphop]

@callmehiphop with your plan, will our library wrap level(up|down) and provide a complete solution for doing lookups, puts, etc? Or do we just offer an adapter that can be plugged into level(up|down) which the user has to manually install?

I think we'd want to just offer the adapter and allow the user to install levelup on their own, that looks to be the levelup way of doing things.

[stephenplusplus]

Should we offer something that's complete, or nbd?

[callmehiphop]

I'd think it's nbd, but that's just me. What are your feelings on it?

[stephenplusplus]

We're trashing this idea and just going with our own API, right?