GiraffaFS / giraffa

Giraffa FileSystem (Slack: giraffa-fs.slack.com)
https://giraffa.ci.cloudbees.com
Apache License 2.0
17 stars 6 forks source link

Investigate how CoprocessorEndpoint can be used instead of piggybacking on put/deletes. #99

Open octo47 opened 9 years ago

octo47 commented 9 years ago

For now Giraffa uses region coprocessors to hook into put/get operations. Taking into account #75 and dynamically typed nature of KeyValue that seems a bit fragile. Did we consider usage of CoprocessorEndpoints? That is a full blown RPC extension for hbase RPC protocol. So we can use rich methods for doing our operations directly without using hacky way to encode/decody/transform keyvalues. https://blogs.apache.org/hbase/entry/coprocessor_introduction

shvachko commented 9 years ago

Are you saying we should use BaseRowProcessorEndpoint as a base class for NamespaceProcessor? I think we were able to utilise endpoints in hbase 0.94, but 0.95 changed coprocessors a lot, and we had to re-implement some parts of hbase RPC to make it pass certain exceptions back to the client. HDFS ClientProtocol API requires to pass some exceptions, like FileNotFoundException, AccessControlException, etc. from the server to the client.

octo47 commented 9 years ago

You are right @shvachko , we need recent HBase to do so. 1.0 or 0.98. So only some preliminary investigation can be made here. Basically we can server modify operations via Endpoint, but read requests can be served directly like they work currently, using regular scans.

octo47 commented 9 years ago

Or we can just implement CoprocessorService without any base classes like NamespaceProcessor implemented.