Port Phoenix to Hbase96

[jeffreyz88]

1) Change POM to use hbase96 binary 2) Add ProtoBuffer support(you need to install ProtoBuffer2.5 locally to generate PB java files from src/main/protobuf/*.proto) 3) Migrate End-Point coprocessor to HBase 96 style 4) Wrap filter into PB Writables in order for them to be RPCed 5) Change KeyValue to Cell in all needed places

[jtaylor-sfdc]

Nice work, @jeffreyz88. Thanks for the sneak peek. Does it run and do the unit test pass yet? What do you think is the best plan to complete it? How about:

• rebase to our Apache git repo master branch (should be checked in by Monday)
• get all unit tests passing

If you just think of Phoenix as an HBase application, going from 0.94 to 0.96 is a huge amount of work. How do they expect to get customers to go through that?

Thanks!

[jeffreyz88]

Phoenix is an anomaly because it use coprocessors, filter & wals which are affected mostly. If an application only use htable interface, the upgrade effort is trivial. While even if a client just recompile without any code change, in some case the performance will suffer. For example, in 0.96 KeyValue class we have the following function to support backward compatibility:

public byte [] getRow() { return CellUtil.cloneRow(this); }

If the client doesn't use Cell interface, it will incur extra byte array copy when uses those functions.

For the remaining work, can we make port-0.96 branch(or another branch) as a development branch so that we can check intermediate changes and multiple people(Nick or others) can work on it?

My next plan it to 1) rebase from master branch 2) fix unit tests so far I haven't run any yet.

Thanks.

[jtaylor-sfdc]

@jeffreyz88 - early next week, we'll switch to the Apache Phoenix git repo where you and Nick have write access (since your both committers). If you could do your rebase on that, we'll be all set. We're not planning on doing any further development on the github.com/forcedotcom/phoenix repo.

Sound ok?

[jeffreyz88]

Sure. I'll defer the rebase then. Thanks.

[jtaylor-sfdc]

In the meantime, how about if you attempt to get one unit test working? I suspect you'll probably run into enough stuff to keep you busy for a couple of days? And whatever changes you make would transfer over to your rebasing anyway.

[jeffreyz88]

Sure.

[apurtell]

If you just think of Phoenix as an HBase application, going from 0.94 to 0.96 is a huge amount of work. How do they expect to get customers to go through that?

Coprocessors are tightly coupled to HBase internals, so Phoenix had to migrate from 0.94 HBase through the 0.96 "singularity" where a number of incompatible changes were made at once to make that sort of pain, as much as possible, a one time only thing.

While the performance loss of an "internals abstraction layer" might always make such a thing a losing proposition, I would like to point out HBASE-4047. With the caveat that we haven't had sufficient interest or time to do that yet, the decoupling implicit in getting the work accomplished could be that very internal abstraction layer, at least for externally hosted coprocessors.

[jeffreyz88]

All end2end tests are passed on my local dev box against the hbase trunk version.

[apurtell]

That's really awesome @jeffreyz88 .

[jtaylor-sfdc]

WHOOT!!! That's fantastic, @jeffreyz88 - excellent work!!!

[jeffreyz88]

Thanks guys! There are still some unit test failures & huge rebase work ahead. Thanks.

[jeffreyz88]

All fixes have been posted here. I'll start to rebase the code with Phoenix master branch(https://github.com/apache/incubator-phoenix) and check in the code into Apache repo. The development branch retires.

Thanks.