Documentation needs to include HBase configuration

[inmyth]

As Data API v2 works with HBase I think the installation documentation really needs to include HBase configuration. I was struggling to get Data API to work with it and I found some issues during my installation (on Ubuntu 16.04 LTS) which held me back for days. Hopefully a clearer instruction can be added to the documentation.

1. Data API requires Thrift interface. The default port is 9090 and it is already set as such in both config files (api and import). Luckily Thrift already comes with HBase installation package so it can be run with something like .../hbase/bin/hbase-daemon.sh start thrift -p 9090 --infoport 9095 where infoport is an option to display Thrift's status UI. You can also check Thrift status from terminal with jps.
2. Data API runs only in TFramedTransport mode. I believe the default mode on HBase is TBufferedTransport. To activate framed transport on HBase, add these lines between configuration tags in hbase-site.xml

    <property>
         <name>hbase.regionserver.thrift.framed</name>
         <value>true</value>
    </property>

3. All tables required for Data API have to be created manually and all those tables must have column family names "d" and "f". These tables can be created by shell-ing into HBase and run the command create '<table_name>', {NAME=>'d'}, {NAME=>'f'} Below are the table names. Note that test_ prefix can be set in Data API config files. Also as I got the tables by observing table not found errors on the console, some tables might still be missing.

   test_account_exchanges
   test_account_offers
   test_account_payments
   test_balance_changes
   test_exchanges
   test_ledgers
   test_lu_account_memos
   test_lu_account_offers_by_sequence
   test_lu_account_transactions
   test_lu_affected_account_transactions
   test_lu_ledgers_by_index
   test_lu_ledgers_by_time
   test_lu_transactions_by_time
   test_memos
   test_payments
   test_transactions
   test_top_markets

[greed0803]

importer throws exception from time to time. Is there any other configuration needed to avoid this. 2660114 [Thread-60] INFO backtype.storm.task.ShellBolt - ShellLog pid:5485, name:transactions unable to save parsedData: 54336|0 2660115 [Thread-60] INFO backtype.storm.task.ShellBolt - ShellLog pid:5485, name:transactions unable to save transaction: 54336|0 2A1FB9AB61B60C0F7DC596CD1DFF2F6F20BCBDC25AB8DDF63965E3CD11BAB53C 2660115 [Thread-60] INFO backtype.storm.task.ShellBolt - ShellLog pid:5485, name:transactions unable to save parsedData: 54336|0 2660115 [Thread-60] INFO backtype.storm.task.ShellBolt - ShellLog pid:5485, name:transactions unable to save transaction: 54336|0 2A1FB9AB61B60C0F7DC596CD1DFF2F6F20BCBDC25AB8DDF63965E3CD11BAB53C

[inmyth]

I don't get this error. However the latest version is buggy. I'm using 2.1.0.

[greed0803]

Are you using node or storm as importer ? because with node i think there's no problem .

[inmyth]

I'm using node.

[greed0803]

Node only import transactions but does not aggregate it.

[yangyang9966]

I have a question ,what it is the version? ex: node.js , hbase ,thrift server ? i hope get the answer,thanks!