search

chiastic-security / phoenix-for-cloudera

Mirror of Apache Phoenix
Apache License 2.0
73 stars 43 forks source link

Phoenix For Cloudera - v4.7 Support #3

Closed shahamit closed 8 years ago

shahamit commented 8 years ago

I have recently started using phoenix with hbase. Since it doesn't work out-of-the-box on CDH, this git repo has been very helpful.

I wanted to know if there is any recent work being done to make the latest phoenix version 4.7 compatible with CDH 5.5.2.

Thanks, Amit.

apurtell commented 8 years ago

is any recent work being done to make the latest phoenix version 4.7 compatible with CDH 5.5.2.

Yes this is something I will be working on soon.

shahamit commented 8 years ago

That's great. I had a question - At a high level what are the code changes made to phoenix to make it work with cloudera? Given the way phoenix is installed (by copy pasting the jars on the hbase master and region servers) what triggered a need to modify phoenix especially since we don't operate phoenix separately.

Thanks!

apurtell commented 8 years ago

There are several differences between Cloudera HBase and Apache HBase in the internal interfaces upon which Phoenix depends: coprocessor, scanner, and tracing APIs for example. The changes needed are largely simple fixups for slight differences, but these are still necessary for successful compilation.

apurtell commented 8 years ago

Still waiting on the 4.7.0 release. Currently at RC6

apurtell commented 8 years ago

I've run into a roadblock.

Phoenix 4.7 depends on Tephra, developed by Cask. Tephra is also not compatible with CDH HBase because of the changes Cloudera has made to various APIs. Other unit tests and ITs pass, but all of the transactional functionality is broken. Teprha needs a port to CDH first. 

Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.OperationWithAttributes.setAttribute(Ljava/lang/String;[B)Lorg/apache/hadoop/hbase/client/OperationWithAttributes;
at co.cask.tephra.hbase10.TransactionAwareHTable.addToOperation(TransactionAwareHTable.java:670)
at co.cask.tephra.hbase10.TransactionAwareHTable.transactionalizeAction(TransactionAwareHTable.java:559)
at co.cask.tephra.hbase10.TransactionAwareHTable.getScanner(TransactionAwareHTable.java:287)
at org.apache.phoenix.iterate.TableResultIterator.initScanner(TableResultIterator.java:120)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:108)
at org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:1)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
apurtell commented 8 years ago

I pushed up what I have so far as https://github.com/chiastic-security/phoenix-for-cloudera/tree/4.7-HBase-1.0-cdh5.5-WIP-BROKEN

apurtell commented 8 years ago

It so happens I heard on good authority that Tephra has a CDH compatibility module, and indeed it does.

Pushed up https://github.com/chiastic-security/phoenix-for-cloudera/tree/4.7-HBase-1.0-cdh5.5