search

Cascading / cascading.hbase

HBase adapters for Cascading
http://www.cascading.org/
10 stars 11 forks source link

Same column name should be allowed to be used in different column families #8

Open andrewnet opened 9 years ago

andrewnet commented 9 years ago

HBase does not have any limit on the column names cross multiple column families. For instance, if you have column family 'cf1' and 'cf2', you can have the same column name in these column families, i.e. 'cf1:col1', 'cf2:col1'.

However, this is not allowed in cascading.hbase. If you try to create such scheme, cascading.hbase throws an exception. This should be allowed since it is a very basic usage.

fs111 commented 9 years ago

Which of the Schemes are you using? What is the error message you are getting?

andrewnet commented 9 years ago

The scheme is created by using the following code:

Fields keyFields = new Fields( "rowkey" ); String[] familyNames = {"cf1", "cf2"}; Fields[] valueFields = new Fields[]{new Fields( "col1" ), new Fields( "col1" )}; Tap hBaseTap = new HBaseTap( "multitable", new HBaseScheme( keyFields, familyNames, valueFields ), SinkMode.REPLACE );

The exception will be something like below:

Exception in thread "main" java.lang.IllegalArgumentException: duplicate field name found: col1 at cascading.tuple.Fields.validate(Fields.java:835) at cascading.tuple.Fields.(Fields.java:632) at cascading.tuple.Fields.join(Fields.java:264) at cascading.tuple.Fields.join(Fields.java:226) at cascading.hbase.HBaseAbstractScheme.setSourceSink(HBaseAbstractScheme.java:31) at cascading.hbase.HBaseScheme.(HBaseScheme.java:100) at com.ibm.mdm.MyHBaseTestReadWorking.main(MyHBaseTestReadWorking.java:51) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)