GravityLabs / HPaste

HBase DSL for Scala with MapReduce support
http://www.gravity.com/technology/
Apache License 2.0
127 stars 32 forks source link

Confused by schema design found in test suite #17

Open alexanderdean opened 10 years ago

alexanderdean commented 10 years ago

I'm confused by the schema design found in the HPaste test suite:

object ExampleSchema extends Schema {

  //There should only be one HBaseConfiguration object per process.  You'll probably want to manage that
  //instance yourself, so this library expects a reference to that instance.  It's implicitly injected into
  //the code, so the most convenient place to put it is right after you declare your Schema.
  implicit val conf = LocalCluster.getTestConfiguration

This approach is tightly-coupling a test Hadoop configuration into the schema object. Obviously this is fine for a project which will never be run on a real cluster, but what's the recommended approach for a schema which will be used "in anger", i.e. needs to support LocalCluster.getTestConfiguration and the real Hadoop cluster's Configuration? (Bearing in mind that implicit values in Scala can't cross object boundaries.)

alexanderdean commented 10 years ago

To workaround this I've changed the schema definition to a class:

/**
 * Holds our HBase schema for lookups into the
 * API Transactions table.
 */
class ApiTransactionSchema(implicit conf: Configuration) extends Schema {

And then select the appropriate Hadoop Configuration in my calling code:

implicit val conf = // Still to write
val ats = new ApiTransactionSchema
val txn = ats.ApiTransactionTable.query2...

I must be missing something much cleaner though - what do you guys do?