I'm fairly new to Hadoop and RHadoop especially. Trying to write a mapreduce function that gets input from HBase (localhost pseudo cluster). Running into a following error:
Be sure to run hdfs.init()
Loading required package: rmr2
Please review your hadoop settings. See help(hadoop.settings)
16/01/13 10:33:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Error: object of type 'closure' is not subsettable
Execution halted
Hi there!
I'm fairly new to Hadoop and RHadoop especially. Trying to write a mapreduce function that gets input from HBase (localhost pseudo cluster). Running into a following error:
Output:
[root@localhost thrift_test]# ./thrift_test.R Loading required package: rhbase Loading required package: methods Loading required package: rhdfs Loading required package: rJava
HADOOP_CMD=/home/hadoop/hadoop/bin/hadoop
Be sure to run hdfs.init() Loading required package: rmr2 Please review your hadoop settings. See help(hadoop.settings) 16/01/13 10:33:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Error: object of type 'closure' is not subsettable Execution halted
Here is the R code:
!/usr/bin/env Rscript
Sys.setenv(HADOOP_CMD="/home/hadoop/hadoop/bin/hadoop") Sys.setenv(HADOOP_STREAMING="/home/hadoop/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.6.2.jar")
require(rhbase) require(rhdfs) require(rmr2)
hdfs.init()
input_format <- make.input.format( format = "hbase", family.columns = list(data = list("execution_id", "automaton_id")), key.deserialize = "raw", cell.deserialize = "raw" )
from.dfs( mapreduce( input = "client1:executions", input.format = input_format, map = function(k, v){v}, verbose = TRUE ) )
Would love to find out what am I doing wrong.
Thank you in advance!